Search Images Maps Play YouTube News Gmail Drive More »
Sign in
Screen reader users: click this link for accessible mode. Accessible mode has the same essential features but works better with your reader.

Patents

  1. Advanced Patent Search
Publication numberUS7055068 B2
Publication typeGrant
Application numberUS 10/205,265
Publication dateMay 30, 2006
Filing dateJul 25, 2002
Priority dateJul 25, 2002
Fee statusPaid
Also published asUS20040019833
Publication number10205265, 205265, US 7055068 B2, US 7055068B2, US-B2-7055068, US7055068 B2, US7055068B2
InventorsDaniel A. Riedl
Original AssigneeLsi Logic Corporation
Export CitationBiBTeX, EndNote, RefMan
External Links: USPTO, USPTO Assignment, Espacenet
Method for validating operation of a fibre link
US 7055068 B2
Abstract
A method for validating operation of a fiber link when the fiber link is initialized includes the steps of entering a trial link up state upon receiving a command to initialize the fiber link so that normal commands to other devices within the fiber channel loop are not resumed, and thereafter entering a final link up state and resuming normal commands to other devices within the fiber channel loop. In exemplary embodiments, the method may be implemented by devices within a system such as a disk array system of a storage area network (SAN), or the like.
Images(4)
Previous page
Next page
Claims(22)
1. A method for validating operation of a fibre link within a fibre channel loop, comprising:
receiving a Loop Initialization Primitive (LIP) to initialize the fibre link;
entering a trial link up state upon receiving the Loop Initialization Primitive (LIP) so that normal commands to other devices within the fibre channel loop are not resumed; and
thereafter entering a final link up state and resuming normal commands to other devices within the fibre channel loop,
wherein a Loop Initialization Primitive (LIP) command is not sent by any device within the fibre channel loop during the trial link up state.
2. The method as claimed in claim 1 wherein the Loop Initialization Primitive (LIP) includes a Loop Initialization Primitive completion frame (LILP).
3. The method as claimed in claim 1, further comprising sending a no operation frame (NOP) while in the trial linkup state, and receiving the no operation frame (NOP) whereupon the final linkup state is entered upon receiving the no operation frame (NOP).
4. The method as claimed in claim 1, further comprising sending a self-directed command while in the trial linkup state and receiving the self-directed command for verifying that the fibre link is stable.
5. The method as claimed in claim 4, wherein the self-directed command comprises at least one of a read command and a write command.
6. A system capable of validating operation of a fibre link of the system, comprising:
means for receiving a command to initialize the fibre link, the command to initialize the fibre link comprising a Loop Initialization Primitive (LIP);
means for entering a trial link up state upon receiving a command to initialize the fibre link so that normal commands to other devices within a fibre channel loop are not resumed; and
means for thereafter entering a final link up state and resuming normal commands to other devices within the fibre channel loop,
wherein a Loop Initialization Primitive (LIP) command is not sent by any device within the fibre channel loop during the trial link up state.
7. The system as claimed in claim 6, wherein the Loop Initialization Primitive (LIP) includes a Loop Initialization Primitive completion frame (LILP).
8. The system as claimed in claim 6, further comprising means for sending a no operation frame (NOP) while in the trial linkup state, and receiving the no operation frame (NOP) whereupon the final linkup state is entered upon receiving the no operation frame (NOP).
9. The system as claimed in claim 6, further comprising means for sending a self-directed command while in the trial linkup state and receiving the self-directed command for verifying that the fibre link is stable.
10. The system as claimed in claim 9, wherein the self-directed command comprises at least one of a read command and a write command.
11. A disk array system, comprising:
a disk array controller for controlling devices within the disk array system; and
a device within the disk array system, the device being coupled to the disk array controller via a fibre channel loop,
wherein at least one of the disk array controller and the device is capable of validating operation of a fibre link within the fibre channel loop by entering a trial link up state upon receiving a command to initialize the fibre link so that normal commands to other devices within the fibre channel loop are not resumed and thereafter entering a final link up state and resuming normal commands to other devices within the fibre channel loop,
wherein a Loop Initialization Primitive (LIP) command is not sent by any device within the fibre channel loop during the trial link up state, and
wherein at least one of the disk array controller and the device sends a no operation frame (NOP) to itself while in the trial linkup state and enters the final linkup state upon receiving the no operation frame (NOP).
12. The disk array system as claimed in claim 11, wherein the disk array controller provided a command to initialize the fibre link to the device.
13. The disk array system as claimed in claim 12, wherein the command to initialize the fibre link comprises a Loop Initialization Primitive (LIP).
14. The disk array system as claimed in claim 13, wherein the Loop Initialization Primitive (LIP) includes a Loop Initialization Primitive completion frame (LILP).
15. The disk array system as claimed in claim 11, wherein at least one of the disk array controller and the device is capable of sending and receiving a self-directed command while in the trial linkup state for verifying that the fibre link is stable.
16. The disk array system as claimed in claim 15, wherein the self-directed command comprises at least one of a read command and a write command.
17. A storage area network, comprising:
a disk array controller for controlling devices within a disk array system; and
a device within the disk array system, the device being coupled to the disk array controller via a fibre link,
wherein at least one of the disk array controller and the device is capable of validating operation of a fibre link by entering a trial link up state upon receiving a command to initialize the fibre link so that normal commands to other devices within a fibre channel loop are not resumed and thereafter entering a final link up state and resuming normal commands to other devices within the fibre channel loop,
wherein a Loop Initialization Primitive (LIP) command is not sent by any device within the fibre channel loop during the trial link up state, and
wherein at least one of the disk array controller and the device sends a no operation frame (NOP) to itself while in the trial linkup state and enters the final linkup state upon receiving the no operation frame (NOP).
18. The storage area network as claimed in claim 17, wherein the disk array controller provides a command to initialize the fibre link to the device.
19. The storage area network as claimed in claim 18, wherein the command to initialize the fibre link comprises a Loop Initialization Primitive (LIP).
20. The storage area network as claimed in claim 19, wherein the Loop Initialization Primitive (LIP) includes a Loop Initialization Primitive completion frame (LILP).
21. The storage area network as claimed in claim 17, wherein at least one of the disk array controller and the device is capable of sending and receiving a self-directed command while in the trial linkup state for verifying that the fibre link is stable.
22. The storage area network as claimed in claim 17, wherein the self-directed command comprises at least one of a read command and a write command.
Description
FIELD OF THE INVENTION

The present invention generally relates to disk arrays utilized in Fibre Channel Storage Area Networks (SANs), or the like, and particularly to a method for validating operation of a fibre channel loop or link within such disk arrays.

BACKGROUND OF THE INVENTION

Fibre channel disk arrays employed by Storage Area Networks (SANs), and the like, typically comprise multiple drive-side fibre channel loops each having a plurality of devices (e.g., up to 127 devices per loop). In typical implementations, each fibre channel loop includes two disk array controllers and multiple drive enclosures, each having a plurality of drives. Each drive enclosure may further include an Enclosure Service Module (ESM) within, which monitors environmental conditions within the enclosure and attempts to ensure that individual drives within the enclosure do not cause the entire loop to become non-functional.

Normally, a fibre channel loop must be initialized when a new device enters the loop or when an error condition is detected on the loop. When the fibre channel loop (also referred to as “link”) goes down to reinitialize for some reason, it normally completes initialization within 15–30 milliseconds. If the link fails to complete reinitialization within 3–5 seconds, the ESMs bypass their “input” external ports and perform diagnostics to determine if all the drives within the enclosure are functioning properly. This diagnostic process is accomplished by bypassing all the drives and then issuing a Loop Initialization Primitive (LIP), which is the command needed to begin initialization. However, the ESMs do not bypass their “output” external ports. Thus, when the ESMs are performing bypassing and LIPing operations, other devices on the loop see the LIP completions, and can falsely determine that the entire link is up when it is not.

When the disk array controller detects that the link is up (i.e., a “link up”), it attempts to resume processing of commands to the devices on the loop. However, if the link is not truly “up”, these commands will fail to complete. When these commands fail to complete, the disk array controller typically seeks to restore operation of the fibre link by again attempting to initialize the link again via a LIP command. However, this approach is normally not successful since it has a tendency to put the link into a perpetual LIP condition from which it may not recover without manual intervention from an operator.

Consequently, it would be desirable to provide a method for validating operation of a fibre link of such disk arrays (i.e., for validating if the link is truly up) so that the link is not put into a perpetual LIP condition when initialized.

SUMMARY OF THE INVENTION

Accordingly, the present invention is directed to a method for validating operation of a fibre link when the fibre link is initialized. In exemplary embodiments, the method includes the steps of entering a trial link up state upon receiving a command to initialize the fibre link so that normal commands to other devices within the fibre channel loop are not resumed, and thereafter entering a final link up state and resuming normal commands to other devices within the fibre channel loop. In exemplary embodiments, the method may be implemented by devices within a system such as a disk array system of a storage area network (SAN), or the like.

It is to be understood that both the forgoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention as claimed. The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate an embodiment of the invention and together with the general description, serve to explain the principles of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

The numerous advantages of the present invention may be better understood by those skilled in the art by reference to the accompanying figures in which:

FIG. 1 is a block diagram illustrating an exemplary fibre channel loop in accordance with the present invention;

FIG. 2 is a block diagram illustrating the fibre channel loop shown in FIG. 1 during normal operation;

FIG. 3 is a block diagram illustrating the fibre channel loop shown in FIGS. 1 and 2 wherein a fibre link is down and a drive enclosure within the fibre channel loop is performing diagnostics; and

FIG. 4 is a flow diagram illustrating an exemplary method for validating operation of a fibre link, such as the fibre link illustrated in FIGS. 1 through 3.

DETAILED DESCRIPTION OF THE INVENTION

Reference will now be made in detail to the presently preferred embodiments of the invention, examples of which are illustrated in the accompanying drawings.

Referring generally now to FIGS. 1 through 3, an exemplary fibre channel loop in accordance with the present invention is described. Fibre channel loop or fibre link 100 includes a disk array controller (DAC) 102 and multiple drive enclosures (three drive enclosures 104108 are shown), each having a plurality of fibre channel drives (ten drives 110114 are shown). Each drive enclosure 104108 may further include an Enclosure Service Module (ESM) 116, which monitors environmental conditions within the enclosure 104108 via the enclosure control bus 118 and attempts to ensure that individual drives 110114 within the enclosure 104108 do not cause the entire loop 100 to become non-functional.

FIG. 2 provides an expanded views of the first drive enclosure 104 during normal operation (i.e., when the fibre link 100 is up). As shown in FIG. 2, no ports within fibre link 100 (e.g., input and output ports 120122) are bypassed. Consequently, self-directed frames or commands issued by disk array controller 102 will succeed.

FIG. 3 depicts the fibre link 100 shown in FIGS. 1 and 2, wherein the fibre link 100 is down because the first drive enclosure 104 is bypassing its input port 120 but not bypassing its output port 122. When disk array controller 102 detects that the link us up (i.e., a “link up”), it attempts to resume processing of commands to the devices 104116 on the loop. However, if the link is not truly “up”, these commands will fail to complete. In the past, when commands issued by the disk array controller failed to complete, the disk array controller typically sought to restore operation of the fibre link by again attempting to initialize the link again via a LIP command. This attempt to initialize the fibre link tended to put the link into a perpetual LIP condition, where many devices on the loop are initiating LIP commands and these LIP commands fail to complete properly due to Enclosure Service Modules (ESMs) leaving and rejoining the main loop to perform diagnostics.

The present invention provides a method for validating operation of a fibre link when the fibre link is initialized. In accordance with this method, two link states: “trial link up” and “final link up” are introduced. When a device 102116 of fibre channel loop 100 receives a LIP completion (LILP) frame (e.g., disk array controller 102), the device (e.g., disk array controller 102) first enters the trial link up state. Thereafter, the device (e.g., disk array controller 102) enters a final link up state. Preferably, normal commands to other devices (e.g., drive enclosures 106 and 108) within fibre channel loop 100 are not resumed until the device enters the final link up state. In order to enter the final link up state, the device sends a “no operation” (NOP) frame to itself via fibre channel loop 100. If the device receives its own NOP, it can safely assume that the fibre link 100 is truly up, and can enter the final link up state. Normal commands to the other devices on the loop 100 may then be resumed.

The device may also employ self-directed commands for diagnostics to ensure that the link is truly up and available for commands to be sent. While in the trial link up state, the device can send, in addition to the NOP, a command such as a read command, a write command, or the like to itself, allowing the device to determine that the loop is not only operational (i.e., the link is “up”) but is also stable.

Referring now to FIG. 4, an exemplary method for validating operation of a fibre link, such as the fibre link illustrated in FIGS. 1 through 3 is described. In the exemplary embodiment shown, the method 200 is initiated, at step 202, when a command to initialize the fibre link is received by a device within the fibre channel loop. The device then enters a trial link up state, at step 204, so that normal commands to other devices within the fibre channel loop are not resumed. In exemplary embodiments, the command to initialize the fibre link comprises a Loop Initialization Primitive (LIP) including a Loop Initialization Primitive completion frame (LILP). While in the trial link up state, the device may send and receive one or more self-directed commands via the fibre link for verifying that the link is stable, at step 206. In exemplary embodiments, these self-directed commands may include read commands, a write commands, or the like. To enter the final link up state, the device sends a no operation frame (NOP) to itself, while in the trial linkup state, at step 208. When the device receives the no operation frame (NOP), the device enters the final linkup state, at step 210, and resumes normal commands to other devices within the fibre channel loop.

The present invention allows the fibre link 100 (FIGS. 1, 2 and 3) to become operable or “come up” without excessive re-initializations (e.g., via LIP commands). Thus, employing the present invention it is typically unnecessary for an operator to manually power-cycle all devices on the fibre channel loop 100 in order to complete re-initialization of the fibre link. Preferably, the present invention may employed by any topology where elements of the whole group can be segmented, which provokes other elements not within that segment to assume that the whole group is functioning properly, and take actions based upon that assumption. For example, in embodiments of the invention illustrated herein, fibre channel loop 100 comprises a loop of a disk array system 130 employed by a storage area network 140, or the like.

In exemplary embodiments, the methods disclosed may be implemented as sets of instructions (e.g., software or firmware) readable by a device within the fibre channel loop 100 (FIGS. 1, 2 and 3). Further, it is understood that the specific order or hierarchy of steps in the methods disclosed are examples of exemplary approaches. Based upon design preferences, it is understood that the specific order or hierarchy of steps in the method can be rearranged while remaining within the scope of the present invention. The accompanying method claims present elements of the various steps in a sample order, and are not meant to be limited to the specific order or hierarchy presented.

It is believed that the present invention and many of its attendant advantages will be understood by the forgoing description. It is also believed that it will be apparent that various changes may be made in the form, construction and arrangement of the components thereof without departing from the scope and spirit of the invention or without sacrificing all of its material advantages. The form herein before described being merely an explanatory embodiment thereof. It is the intention of the following claims to encompass and include such changes.

Patent Citations
Cited PatentFiling datePublication dateApplicantTitle
US5978379 *Jan 23, 1997Nov 2, 1999Gadzoox Networks, Inc.Fiber channel learning bridge, learning half bridge, and protocol
US6138161 *Feb 18, 1999Oct 24, 2000Crossroads Systems, Inc.Method and system for maintaining reserve command relationships in a fibre channel network
US6289002 *Sep 15, 1999Sep 11, 2001Emulex CorporationAutomatic isolation in loops
US6430714 *Aug 6, 1999Aug 6, 2002Emc CorporationFailure detection and isolation
US6438285 *May 31, 2000Aug 20, 2002International Business Machines CorporationFacility for intializing a fiber optic data link in one mode of a plurality of modes
US6609165 *Sep 27, 1999Aug 19, 2003International Business Machines CorporationMethod and apparatus for using fibre channel extended link service commands in a point-to-point configuration
US6748510 *Feb 28, 2002Jun 8, 2004Network Appliance, Inc.System and method for verifying disk configuration
US6915429 *Nov 16, 2000Jul 5, 2005International Business Machines CorporationSystem and method for fiber channel remote activation and enhanced security
US6917763 *Mar 5, 2002Jul 12, 2005Nortel Networks LimitedTechnique for verifying fiber connectivity in a photonic network
US20020002440 *Jun 19, 2001Jan 3, 2002Nec CorporationLoop diagnosis system and method for disk array apparatuses
US20020010881 *Jun 1, 2001Jan 24, 2002White James L.Method and system for enhancing fibre channel loop resiliency for a mass storage enclosure by increasing component redundancy and using shunt elements and intelligent bypass management
US20020067738 *Dec 4, 2000Jun 6, 2002Sam SuOld-port node detection and hub port bypass
US20020089713 *Nov 26, 2001Jul 11, 2002Alfred SchwandnerMethod and device for checking the operativeness of an optical transmission link
US20020152338 *Aug 27, 2001Oct 17, 2002Elliott Joseph C.Method, system and program product for detecting lost sequences within an exchange on fibre channel
US20030056153 *Jun 14, 2002Mar 20, 2003International Business Machines CorporationMethod and apparatus for fault location in a loop network
US20030212785 *May 8, 2002Nov 13, 2003Jibbe Mahmoud K.System and method for isolating faulty connections in a storage area network
Referenced by
Citing PatentFiling datePublication dateApplicantTitle
US7334046Aug 5, 2002Feb 19, 2008Qlogic, CorporationSystem and method for optimizing frame routing in a network
US7340167Oct 8, 2004Mar 4, 2008Qlogic, CorporationFibre channel transparent switch for mixed switch fabrics
US7352701Sep 19, 2003Apr 1, 2008Qlogic, CorporationBuffer to buffer credit recovery for in-line fibre channel credit extension devices
US7355966 *Jul 12, 2004Apr 8, 2008Qlogic, CorporationMethod and system for minimizing disruption in common-access networks
US7362717Oct 3, 2002Apr 22, 2008Qlogic, CorporationMethod and system for using distributed name servers in multi-module fibre channel switches
US7404020Jul 20, 2004Jul 22, 2008Qlogic, CorporationIntegrated fibre channel fabric controller
US7661017Jan 30, 2007Feb 9, 2010International Business Machines CorporaionDiagnostic operations within a switched fibre channel arbitrated loop system
Classifications
U.S. Classification714/43, 370/222, 370/241, 714/717, 398/13, 398/59
International ClassificationG06F11/00
Cooperative ClassificationG06F13/4221
European ClassificationG06F13/42C2
Legal Events
DateCodeEventDescription
Apr 3, 2015ASAssignment
Owner name: AVAGO TECHNOLOGIES GENERAL IP (SINGAPORE) PTE. LTD
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:LSI CORPORATION;REEL/FRAME:035390/0388
Effective date: 20140814
Jun 6, 2014ASAssignment
Owner name: LSI CORPORATION, CALIFORNIA
Free format text: CHANGE OF NAME;ASSIGNOR:LSI LOGIC CORPORATION;REEL/FRAME:033102/0270
Effective date: 20070406
May 8, 2014ASAssignment
Owner name: DEUTSCHE BANK AG NEW YORK BRANCH, AS COLLATERAL AG
Free format text: PATENT SECURITY AGREEMENT;ASSIGNORS:LSI CORPORATION;AGERE SYSTEMS LLC;REEL/FRAME:032856/0031
Effective date: 20140506
Oct 30, 2013FPAYFee payment
Year of fee payment: 8
Nov 26, 2009FPAYFee payment
Year of fee payment: 4
Jul 25, 2002ASAssignment
Owner name: LSI LOGIC CORPORATION, CALIFORNIA
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:RIEDL, DANIEL A.;REEL/FRAME:013151/0689
Effective date: 20020722