WO2007111664A2 - Fault resilient boot in multi-processor systems - Google Patents
Fault resilient boot in multi-processor systems Download PDFInfo
- Publication number
- WO2007111664A2 WO2007111664A2 PCT/US2006/048295 US2006048295W WO2007111664A2 WO 2007111664 A2 WO2007111664 A2 WO 2007111664A2 US 2006048295 W US2006048295 W US 2006048295W WO 2007111664 A2 WO2007111664 A2 WO 2007111664A2
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- processor
- boot
- failure
- application
- progress
- Prior art date
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
- G06F11/0706—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation the processing taking place on a specific hardware platform or in a specific software environment
- G06F11/0721—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation the processing taking place on a specific hardware platform or in a specific software environment within a central processing unit [CPU]
- G06F11/0724—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation the processing taking place on a specific hardware platform or in a specific software environment within a central processing unit [CPU] in a multiprocessor or a multi-core unit
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1415—Saving, restoring, recovering or retrying at system level
- G06F11/1417—Boot up procedures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
- G06F11/0751—Error or fault detection not based on redundancy
- G06F11/0754—Error or fault detection not based on redundancy by exceeding limits
- G06F11/0757—Error or fault detection not based on redundancy by exceeding limits by exceeding a time limit, i.e. time-out, e.g. watchdogs
Definitions
- the inventions generally relate to fault resilient boot in multi-processor systems.
- Reliable system boot is a significant RAS (Reliability Availability Serviceability) feature in multi-processor platforms.
- a separate service processor is provided on the platform to select the system boot strap processor and to ensure that the system boots.
- the service processor is responsible for disabling the failed processor and selecting an alternative processor. This process is known as a "Fault Resilient Boot” (FRB).
- FTB fault Resilient Boot
- FIG 1 illustrates a multi-processor (MP) system according to some embodiments of the inventions.
- FIG 2 illustrates a flowchart according to some embodiments of the inventions.
- Some embodiments of the inventions relate to fault resilient boot in multiprocessor systems.
- a boot progress of a System Boot Strap Processor in a multi-processor system is monitored and a boot processor failure is detected using an Application Processor. If the boot processor failure is detected at least a portion of the system is reinitialized (and/or the entire system is rebooted).
- a system includes a System Boot Strap Processor and an Application Processor to monitor a boot progress of the System Boot Strap Processor, to detect a boot processor failure, and to reinitialize at least a portion of the system (and/or reboot the entire system) if the boot processor failure is detected.
- a system includes at a minimum a first processor and a second processor.
- One processor of the system becomes a System Boot Strap Processor, and all other processor of the system become Application Processors.
- At least one Application Processor is to monitor a boot progress of the System Boot Strap Processor, to detect a boot processor failure, and to reinitialize at least a portion of the system (and/or reboot the system) if the boot processor failure is detected.
- an article includes a computer readable medium having instructions thereon which when executed cause a computer to monitor a boot progress of a System Boot Strap Processor in a multi-processor system using an Application Processor, detect a boot processor failure using the Application Processor, and reinitialize at least a portion of the system (and/or reboot the entire system) if the boot processor failure is detected.
- a firmware-based solution is used to implement fault resilient booting without any requirement for a service processor.
- the firmware may be Basic Input/Output System (BIOS), on-package firmware or microcode. According to some embodiments, this solution may even be implemented on low end dual processor (DP) server platforms.
- CSI high-speed interconnect
- the CSI link interconnects will be used more and more as multi-core, multithreaded processors become more popular.
- Link based architecture such as CSI allows for system partitioning. However, it may be impractical to implement a service processor for every possible system partition. Therefore, fault resilient booting according to some embodiments is very advantageous because it does not require a service processor.
- system partitions can implement fault resilient boot (FRB) without requiring a per partition service processor.
- FLB fault resilient boot
- FIG 1 illustrates a multi-processor (MP) system 100 according to some embodiments.
- System 100 includes a first processor (and/or processor socket) 102 and a second processor (and/or processor socket) 104.
- Processor 102 is a multi- core (MC) processor including a first core 106 and a second core 108.
- processor 104 is a multi-core (MC) processor including a first core 112 and a second core 114.
- System 100 also includes a chipset 122 coupled to four PCI Express buses 124, 126, 128, 130.
- An I/O Controller Hub (ICH) 132 is also coupled to chipset 122.
- Processor 102 and processor 104 are each coupled to chipset 122 via a respective interconnect (for example, a high speed interconnect such as a CSI link).
- any link- based architecture may be used (for example, any link-based architecture such as a multiple front side bus architecture, CSI, hypertransport, etc.)
- a Fault Resilient Boot may be implemented without requiring a service processor.
- MP multi-processor
- SBSP System Boot Strap Processor
- the idling processors may be taken advantage of to implement a fault resilient boot (FRB).
- the non-boot strap processors commonly referred to as Application processors or APs
- APs Application processors
- the processor package (or processor core) that first writes its Local APIC (Advanced Programmable Interrupt Controller) ID to a chipset control register becomes the System Boot Strap Processor (SBSP).
- SBSP System Boot Strap Processor
- All other cores than the SBSP become an Application Processor (AP).
- the AP can now read the chipset reset register to detect which core became the SBSP. Instead of putting the AP to sleep during the initialization of the system by the SBSP, the AP starts monitoring the SBSP boot progress (for example, as outlined by the flowchart in FIG 2).
- FIG 2 illustrates a flowchart 200 according to some embodiments.
- a system reset occurs.
- the System Boot Strap Processor (SBSP) is chosen (for example, the first processor package (core) that writes its Local APIC ID to a chipset control register). Previously disabled processors (or cores) will not participate in the SBSP selection.
- SBSP System Boot Strap Processor
- Previously disabled processors or cores will not participate in the SBSP selection.
- a determination is made as to whether the current processor core is the SBSP. If it is the SBSP at 206 the SBSP makes a determination at 208 as to whether the SBSP's health is good (for example, by checking Built in Self Test results). If the health is good at 208, then the SBSP continues with the boot at 210.
- the SBSP also indicates at 210 the boot progress at critical checkpoints of the boot process in a chipset Scratch Register (SR) that is readable by the APs. If the health of the SBSP is not good at 208 then the health status of the SBSP is indicated at 212, and the SBSP waits for the monitoring AP to disable the SBSP.
- SR chipset Scratch Register
- each AP checks its health (for example, according to some embodiments by checking the Built in Self Test (BIST) results). If the AP is not healthy to execute at 214, the AP disables itself at 216 from participating in the monitoring of the SBSP boot process. APs that are healthy at 214 maintain a boot time elapse counter by reading the processor/chipset specific Interval Timer Counter (ITC) at 218, and setting the start time variable. The APs periodically check the SBSP boot progress (for example, by checking the chipset Scratch Register) against the elapsed timer counter to determine the SBSP progress status at 224.
- ITC Interval Timer Counter
- the AP logs at 228 the identity of the current SBSP to a sticky register in each processor package (core), and then reinitializes at least a portion of the system (and/or reboots the system).
- each processor will check to see if it failed as the SBSP in the previous boot by referring to its sticky register. If so, it will not attempt to become the SBSP, and instead will disable itself. If the SBSP boot progresses sufficiently (as indicated by a write to a chipset register late in the boot process), the APs will stop monitoring the SBSP boot progress and will either return to idling or to other activity determined by system boot. If the AP is requested to join the boot at 230, the AP will join the system boot at 232.
- each system shown in a figure the elements in some cases may each have a same reference number or a different reference number to suggest that the elements represented could be different and/or similar.
- an element may be flexible enough to have different implementations and work with some or all of the systems shown or described herein.
- the various elements shown in the figures may be the same or different. Which one is referred to as a first element and which is called a second element is arbitrary.
- Coupled may mean that two or more elements are in direct physical or electrical contact. However, “coupled” may also mean that two or more elements are not in direct contact with each other, but yet still co-operate or interact with each other.
- An algorithm is here, and generally, considered to be a self-consistent sequence of acts or operations leading to a desired result. These include physical manipulations of physical quantities. Usually, though not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated. It has proven convenient at times, principally for reasons of common usage, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers or the like. It should be understood, however, that all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities.
- Some embodiments may be implemented in one or a combination of hardware, firmware, and software. Some embodiments may also be implemented as instructions stored on a machine-readable medium, which may be read and executed by a computing platform to perform the operations described herein.
- a machine-readable medium may include any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer).
- a machine-readable medium may include read only memory (ROM); random access memory (RAM); magnetic disk storage media; optical storage media; flash memory devices; electrical, optical, acoustical or other form of propagated signals (e.g., carrier waves, infrared signals, digital signals, the interfaces that transmit and/or receive signals, etc.), and others.
- An embodiment is an implementation or example of the inventions.
- Reference in the specification to "an embodiment,” “one embodiment,” “some embodiments,” or “other embodiments” means that a particular feature, structure, or characteristic described in connection with the embodiments is included in at least some embodiments, but not necessarily all embodiments, of the inventions.
- the various appearances “an embodiment,” “one embodiment,” or “some embodiments” are not necessarily all referring to the same embodiments.
Abstract
Description
Claims
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
DE112006003598T DE112006003598T5 (en) | 2005-12-30 | 2006-12-18 | Fault-tolerant booting in multiprocessor systems |
CN200680045005XA CN101322104B (en) | 2005-12-30 | 2006-12-18 | Fault resilient boot in multi-processer system |
GB0809459A GB2446094B (en) | 2005-12-30 | 2006-12-18 | Fault resilient boot in multi-processor systems |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/322,997 | 2005-12-30 | ||
US11/322,997 US7472266B2 (en) | 2005-12-30 | 2005-12-30 | Fault resilient boot in multi-processor systems |
Publications (2)
Publication Number | Publication Date |
---|---|
WO2007111664A2 true WO2007111664A2 (en) | 2007-10-04 |
WO2007111664A3 WO2007111664A3 (en) | 2008-01-24 |
Family
ID=38226048
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US2006/048295 WO2007111664A2 (en) | 2005-12-30 | 2006-12-18 | Fault resilient boot in multi-processor systems |
Country Status (5)
Country | Link |
---|---|
US (1) | US7472266B2 (en) |
CN (1) | CN101322104B (en) |
DE (1) | DE112006003598T5 (en) |
GB (1) | GB2446094B (en) |
WO (1) | WO2007111664A2 (en) |
Families Citing this family (33)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8478811B2 (en) | 2002-10-08 | 2013-07-02 | Netlogic Microsystems, Inc. | Advanced processor with credit based scheme for optimal packet flow in a multi-processor system on a chip |
US7961723B2 (en) | 2002-10-08 | 2011-06-14 | Netlogic Microsystems, Inc. | Advanced processor with mechanism for enforcing ordering between information sent on two independent networks |
US7334086B2 (en) | 2002-10-08 | 2008-02-19 | Rmi Corporation | Advanced processor with system on a chip interconnect technology |
US7346757B2 (en) | 2002-10-08 | 2008-03-18 | Rmi Corporation | Advanced processor translation lookaside buffer management in a multithreaded system |
US7984268B2 (en) | 2002-10-08 | 2011-07-19 | Netlogic Microsystems, Inc. | Advanced processor scheduling in a multithreaded system |
US8015567B2 (en) | 2002-10-08 | 2011-09-06 | Netlogic Microsystems, Inc. | Advanced processor with mechanism for packet distribution at high line rate |
US8176298B2 (en) * | 2002-10-08 | 2012-05-08 | Netlogic Microsystems, Inc. | Multi-core multi-threaded processing systems with instruction reordering in an in-order pipeline |
US7924828B2 (en) | 2002-10-08 | 2011-04-12 | Netlogic Microsystems, Inc. | Advanced processor with mechanism for fast packet queuing operations |
US9088474B2 (en) | 2002-10-08 | 2015-07-21 | Broadcom Corporation | Advanced processor with interfacing messaging network to a CPU |
US7627721B2 (en) | 2002-10-08 | 2009-12-01 | Rmi Corporation | Advanced processor with cache coherency |
US8037224B2 (en) | 2002-10-08 | 2011-10-11 | Netlogic Microsystems, Inc. | Delegating network processor operations to star topology serial bus interfaces |
US7987352B2 (en) * | 2007-11-30 | 2011-07-26 | Intel Corporation | Booting with sub socket partitioning |
US9760424B2 (en) * | 2008-01-31 | 2017-09-12 | Thomson Licensing Dtv | Systems and methods for dynamically reporting a boot process in content/service receivers |
US9596324B2 (en) | 2008-02-08 | 2017-03-14 | Broadcom Corporation | System and method for parsing and allocating a plurality of packets to processor core threads |
US7971098B2 (en) * | 2008-03-24 | 2011-06-28 | Globalfoundries Inc. | Bootstrap device and methods thereof |
US7941699B2 (en) * | 2008-03-24 | 2011-05-10 | Intel Corporation | Determining a set of processor cores to boot |
JP5460167B2 (en) * | 2009-07-31 | 2014-04-02 | キヤノン株式会社 | Information processing apparatus, control method for information processing apparatus, and control program |
JP5516747B2 (en) * | 2010-10-05 | 2014-06-11 | 富士通株式会社 | Multi-core processor system, supervisory control method, and supervisory control program |
EP2656227A2 (en) * | 2010-12-22 | 2013-10-30 | Intel Corporation | Debugging complex multi-core and multi-socket systems |
CN102043690A (en) * | 2010-12-31 | 2011-05-04 | 上海华为技术有限公司 | Fault-handling method for multi-core processor and multi-core processor |
US8850177B2 (en) * | 2011-07-08 | 2014-09-30 | Openpeak Inc. | System and method for validating components during a booting process |
ITMI20111287A1 (en) | 2011-07-11 | 2013-01-12 | Ibm | DISTRIBUTIONS OF OPERATING SYSTEMS WITH DETECTION OF CYCLE CONDITIONS |
US8954721B2 (en) | 2011-12-08 | 2015-02-10 | International Business Machines Corporation | Multi-chip initialization using a parallel firmware boot process |
US8914458B2 (en) * | 2012-09-27 | 2014-12-16 | Mellanox Technologies Ltd. | Look-ahead handling of page faults in I/O operations |
US9639464B2 (en) | 2012-09-27 | 2017-05-02 | Mellanox Technologies, Ltd. | Application-assisted handling of page faults in I/O operations |
US9135126B2 (en) * | 2013-02-07 | 2015-09-15 | International Business Machines Corporation | Multi-core re-initialization failure control system |
US20150294119A1 (en) * | 2014-04-10 | 2015-10-15 | International Business Machines Corporation | Booting a multi-node computer system from a primary node dynamically selected based on security setting criteria |
US10031857B2 (en) | 2014-05-27 | 2018-07-24 | Mellanox Technologies, Ltd. | Address translation services for direct accessing of local memory over a network fabric |
US10120832B2 (en) | 2014-05-27 | 2018-11-06 | Mellanox Technologies, Ltd. | Direct access to local memory in a PCI-E device |
WO2016075699A1 (en) * | 2014-11-13 | 2016-05-19 | Hewlett-Packard Development Company, L.P. | Dual purpose boot registers |
CN106844082A (en) * | 2017-01-18 | 2017-06-13 | 联想(北京)有限公司 | Processor predictive failure analysis method and device |
US10628275B2 (en) * | 2018-03-07 | 2020-04-21 | Nxp B.V. | Runtime software-based self-test with mutual inter-core checking |
US20230393935A1 (en) * | 2022-06-02 | 2023-12-07 | Micron Technology, Inc. | Evaluation of memory device health monitoring logic |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5951683A (en) * | 1994-01-28 | 1999-09-14 | Fujitsu Limited | Multiprocessor system and its control method |
US20030005275A1 (en) * | 2001-06-19 | 2003-01-02 | Lam Son H. | Fault resilient booting for multiprocessor system using appliance server management |
US20030009654A1 (en) * | 2001-06-29 | 2003-01-09 | Nalawadi Rajeev K. | Computer system having a single processor equipped to serve as multiple logical processors for pre-boot software to execute pre-boot tasks in parallel |
US20030163765A1 (en) * | 1998-12-29 | 2003-08-28 | Donald J. Eckardt | Method and apparatus for providing diagnosis of a processor without an operating system boot |
Family Cites Families (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6058475A (en) * | 1997-09-22 | 2000-05-02 | Ncr Corporation | Booting method for multi-processor computer |
US6601165B2 (en) * | 1999-03-26 | 2003-07-29 | Hewlett-Packard Company | Apparatus and method for implementing fault resilient booting in a multi-processor system by using a flush command to control resetting of the processors and isolating failed processors |
-
2005
- 2005-12-30 US US11/322,997 patent/US7472266B2/en not_active Expired - Fee Related
-
2006
- 2006-12-18 DE DE112006003598T patent/DE112006003598T5/en not_active Withdrawn
- 2006-12-18 WO PCT/US2006/048295 patent/WO2007111664A2/en active Application Filing
- 2006-12-18 CN CN200680045005XA patent/CN101322104B/en not_active Expired - Fee Related
- 2006-12-18 GB GB0809459A patent/GB2446094B/en not_active Expired - Fee Related
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5951683A (en) * | 1994-01-28 | 1999-09-14 | Fujitsu Limited | Multiprocessor system and its control method |
US20030163765A1 (en) * | 1998-12-29 | 2003-08-28 | Donald J. Eckardt | Method and apparatus for providing diagnosis of a processor without an operating system boot |
US20030005275A1 (en) * | 2001-06-19 | 2003-01-02 | Lam Son H. | Fault resilient booting for multiprocessor system using appliance server management |
US20030009654A1 (en) * | 2001-06-29 | 2003-01-09 | Nalawadi Rajeev K. | Computer system having a single processor equipped to serve as multiple logical processors for pre-boot software to execute pre-boot tasks in parallel |
Non-Patent Citations (1)
Title |
---|
"Intel server platform SR8770BH2 Field Error Reference Guide" INTEL ENTERPRISE PLATFORMS AND SERVICE DIVISION, March 2004 (2004-03), XP002458626 * |
Also Published As
Publication number | Publication date |
---|---|
GB2446094A (en) | 2008-07-30 |
US7472266B2 (en) | 2008-12-30 |
GB2446094B (en) | 2011-08-03 |
US20070157011A1 (en) | 2007-07-05 |
WO2007111664A3 (en) | 2008-01-24 |
DE112006003598T5 (en) | 2008-11-13 |
CN101322104B (en) | 2012-05-30 |
CN101322104A (en) | 2008-12-10 |
GB0809459D0 (en) | 2008-07-02 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7472266B2 (en) | Fault resilient boot in multi-processor systems | |
JP3461825B2 (en) | Multiprocessor distributed initialization and self-test system | |
US5491788A (en) | Method of booting a multiprocessor computer where execution is transferring from a first processor to a second processor based on the first processor having had a critical error | |
US8898517B2 (en) | Handling a failed processor of a multiprocessor information handling system | |
US11068360B2 (en) | Error recovery method and apparatus based on a lockup mechanism | |
US10713128B2 (en) | Error recovery in volatile memory regions | |
US20110179311A1 (en) | Injecting error and/or migrating memory in a computing system | |
US7984219B2 (en) | Enhanced CPU RASUM feature in ISS servers | |
US7783872B2 (en) | System and method to enable an event timer in a multiple event timer operating environment | |
US11789820B2 (en) | Methods and systems for preventing hangup in a post routine from faulty bios settings | |
US7434102B2 (en) | High density compute center resilient booting | |
US6725396B2 (en) | Identifying field replaceable units responsible for faults detected with processor timeouts utilizing IPL boot progress indicator status | |
US8898653B2 (en) | Non-disruptive code update of a single processor in a multi-processor computing system | |
EP3698251A1 (en) | Error recovery in non-volatile storage partitions | |
US7991933B2 (en) | Synchronizing processors when entering system management mode | |
US8078858B2 (en) | Manufacturing boot process with minimal component initialization | |
US20210349731A1 (en) | Booting and using a single cpu socket as a multi-cpu partitioned platform | |
US10824493B2 (en) | Disambiguation of error logging during system reset | |
US11366710B1 (en) | Methods and systems for reducing downtime from system management mode in a computer system | |
US20080313489A1 (en) | Flash memory-hosted local and remote out-of-service platform manageability | |
WO2014158161A1 (en) | Error coordination message for a blade device having a logical processor in another system firmware domain | |
US10768940B2 (en) | Restoring a processing unit that has become hung during execution of an option ROM | |
US11487621B1 (en) | Linking embedded controller with memory reference code and system bios shadowing | |
US20240103971A1 (en) | Systems and methods for error recovery in rebootless firmware updates |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
WWE | Wipo information: entry into national phase |
Ref document number: 200680045005.X Country of ref document: CN |
|
ENP | Entry into the national phase |
Ref document number: 0809459 Country of ref document: GB Kind code of ref document: A Free format text: PCT FILING DATE = 20061218 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 0809459.1 Country of ref document: GB |
|
WWE | Wipo information: entry into national phase |
Ref document number: 1120060035983 Country of ref document: DE |
|
RET | De translation (de og part 6b) |
Ref document number: 112006003598 Country of ref document: DE Date of ref document: 20081113 Kind code of ref document: P |
|
WWE | Wipo information: entry into national phase |
Ref document number: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 06849977 Country of ref document: EP Kind code of ref document: A2 |