US 20050203989 A1
A high speed, microcomputer based, Fibre Channel compatible and fault tolerant information processing and mass storage system especially suited for information servers and application servers. A unique and extremely versatile system architecture, including a dual loop arbitrated, Fibre Channel capable, multiple-fault tolerant, hot swappable mass storage disk array, permits combinations of servers and mass storage arrays which can be tailored for a wide variety of applications and which can be configured with emphasis on the system characteristics such as redundancy, speed, processing capability, storage capability, and the like, as desired. A unique backplane and/or midplane arrangement for connecting the system components allows for easy and, in most cases, on-line field upgrading and/or service and at the same time provides for the very effective cooling of components, particularly those such as disk drives which tend to produce a lot of heat.
1. A method for transferring data in a network server system, the network server system being connected to a network for providing subscribers on a network with access to data from the network server system, comprising:
providing at least one mass storage device;
providing a mass storage device controller, said mass storage device controller being connected to said at least one mass storage device for controlling input and output of said at least one mass storage device;
providing a communications interface, said communications interface providing a connection to the network for receiving input signals from the network and for outputting signals from the network server system to the network, said mass storage device controller being connected to said communications interface for receiving input signals from the network and for outputting signals from said at least one mass storage device to the network;
establishing direct communication between said mass storage device controller and said communications interface; and
maintaining the direct communication between said mass storage device controller and said communications interface to provide the subscribers on the network with access to data from the network server system.
2. The method of
3. The method of
4. The method of
5. A network server system for transferring data, the network server system being connected to a network for providing subscribers on the network with access to data from the network server system, comprising:
at least one mass storage device;
a mass storage device controller, said mass storage device controller being connected to said at least one mass storage device for controlling input and output of said at least one mass storage device;
a communications interface, said communications interface providing a connection to the network for receiving input signals from the network and for outputting signals from the network server system to the network, said mass storage device controller being connected to said communications interface for receiving input signals from the network and for outputting signals from said at least one mass storage device to the network;
means for establishing direct communication between said mass storage device controller and said communications interface; and
means for maintaining the direct communication between said mass storage device controller and said communications interface to provide the subscribers on the network with access to data from the network server system.
6. The network server system of
7. The network server system of
8. The network server system of
9. A network server system, comprising:
a central processing unit;
a mass storage device;
a first controller, said first controller being connected to said mass storage device for controlling input and output of said mass storage device;
a communications interface, said communications interface providing a connection to the network for receiving input signals from the network and for outputting signals from the network server system to the network; and
a second controller communicatively coupled to the central processing unit and the first controller, the second controller communicatively coupled to said communications interface to communicate with a network, the central processing unit being operative to establish direct communication between the first and second controllers, and said first and second controllers being operative to maintain the direct communication independent of the central processing unit.
10. The network server system of
11. A network server system connected to a network for providing subscribers on the network with access to data from the network server system, comprising:
a central processing unit;
a network communications interface;
a mass storage device; and
a storage device controller communicatively coupled to the network communications interface and to the mass storage device to control communications between network communications interface and the mass storage device, said central processing unit being operative to establish direct communications between the mass storage device and the network communications interface via the storage device controller to provide the subscribers on the network with access to data from the network server system.
This application is a divisional of Ser. No. 09/592,572, filed Jun. 12, 2000, and is related to Ser. No. 11/051,852, filed Feb. 4, 2005. This application is related to Patent Application No. PCT/US99/05231 of Richard Dellacona filed Mar. 10, 1999, Ser. No. 09/623,322, now Pat. No. 6,799,224, which is based upon Provisional Patent Application Ser. No. 60/077,643, filed Mar. 10, 1998, and to U.S. patent application Ser. No. 09/071,282 of Richard Dellacona filed May 1, 1998, now Pat. No. 6,260,155, all of which are hereby incorporated herein by reference.
1. Field of the Invention
This invention relates to a high speed, microcomputer based, Fibre Channel compatible and fault tolerant information processing and mass storage system especially suited for information servers and application servers. In particular, the present invention relates to a method and apparatus for information processing and storage involving a unique and extremely versatile system architecture, including a dual loop arbitrated, Fibre Channel capable, multiple-fault tolerant, hot-swappable mass storage disk array and including a method and apparatus for providing an enterprise wide information or application server system using such disk array.
2. State of the Art
Efforts have been made in the past to provide a mass storage file server capable of delivering information throughout an enterprise with high speed data throughput, scalable data storage capability in a convenient, easily configurable enclosure using well known, industry standard operating software and hardware. However, such systems have typically experienced many shortcomings and problems associated with equipment incompatibility as well as with the inability of presently available computer and communications hardware to sustain performance and service failure of component devices. Such shortcomings have included the lack of capability to add storage devices to accommodate increased storage requirements or to replace failed storage devices without the need to completely power down the information server. Some of the compatibility problems have involved, for example, bottlenecks in sharing information among the equipment components of various vendors. The above-referenced Dellacona patent applications address some of these problems and others, and provide unique solutions which are described and claimed therein.
The present invention further addresses some of the problems discussed in the referenced Dellacona applications, as well as others. For example, the present invention further addresses the problem of scalability and customization in information processing and storage systems for different applications. In some applications, there may be a greater need for processing capability rather than storage capacity, while other applications may require just the opposite. Yet other applications may require storage expansion for existing information processing systems. This invention provides an architecture which will readily accommodate such needs.
In addition, mass storage systems can create considerable heat, particularly where they are disk drive based. If the heat is not effectively removed, it can affect the reliability and life of the system. Often, it is difficult to remove the heat because of obstructions caused by the physical configuration of back planes and mid planes which act as barriers to air flow. Typically, for example, all of the disk drives of a mass storage module or array are typically plugged into connectors on the face of a backplane or mid plane that extends across the entire module. Whether enclosed in a cabinet or other enclosure or rack mounted without an enclosure, air flow through the module is inhibited by this type of structural arrangement, and excessive heating can occur, particularly in the vicinity of the disk drives.
Also, it is desirable to be able to hot swap individual disk drives of a mass storage module to accommodate the need for more storage capacity, but the system storage requirements may outgrow the capacity of the module and it may also be desirable to have the capability of adding modules without powering down the system. Of course, this capability must be provided without disturbing the operation of the existing module and with a minimum of signal degradation as modules are added.
The present invention obviates one or more of the foregoing problems and/or shortcomings of the prior art through the provision of an information processing and mass storage method and system, including a unique mass storage array, particularly suited for information servers or application servers, with a novel system architecture which permits the addition or replacement of storage devices without interrupting or seriously degrading the operation of the system and which is highly fault-tolerant and reliable. In addition, the invention obviates one or more of the foregoing problems by providing a novel physical layout that permits the effective removal of heat from a system module containing heat creating components.
In accordance with one embodiment of the invention, an information processing and mass storage system adapted to be readily expandable to increase its storage capacity while the system is in operation comprises at least one module containing (a) at least one computer, (b) a plurality of plug-in storage devices such as disk drives for storing information, (c) a storage device bypass circuit board associated with each storage device, with each storage device being plugged into a connector on the bypass circuit board, (d) a module bypass circuit board including an optical input/output connector for outputting electrical signals from the module as light signals and for inputting light signals into the module as electrical signals and (e) a controller connecting the computer with each of the storage devices through its associated storage device bypass circuit board and through the module bypass circuit board.
Certain information server configurations in accordance with the invention include one or more computers, each computer connected to communicate through a Fibre Channel controller with a mass storage array comprising a plurality of bypass circuit boards, at least some of which are connected to an information storage device. In one embodiment, the controller provides a dual loop communications channel comprising two complete communication paths to each of bypass circuit boards and associated storage devices. In another embodiment, the controller provides a single loop which traverses the bypass circuit boards and any associated storage devices twice. In one configuration with two or more computers, the Fibre Channel controller connected to each computer communicates with the mass storage array through a Fibre Channel controller bypass card.
In a preferred embodiment the computer is preferably a suitable conventional single board computer. The controller preferably is a conventional arbitrated dual channel Fibre Channel system through which the computer communicates with each of the storage device bypass circuit boards and the module bypass circuit board. The bypass circuit boards may be any suitable circuits which form a continuous loop for the Fiber Channel controller regardless of whether a disk drive is plugged into the drive bypass circuit board. The loop continues through other modules when they are connected to the module bypass circuit thereby readily permitting expansion while maintaining a unitary information processing and mass storage system.
In accordance with another embodiment of the present invention, a high speed information processing and mass storage system includes two modules, each including a plurality of disk drives in a hot-swappable, disk drive array. Each disk drive array is connected to a module bypass circuit which includes an optical input/output connector, preferably an optoelectronic transceiver. The optical input/output connectors of the modules are connected by a fiber optic transmission medium such that signals are communicated between the modules in the form of light. With this configuration, modules may be added to increase storage capacity without interrupting the operation of each other and without serious signal degradation.
In accordance with yet another embodiment of the present invention, a high speed information processing and/or mass storage system with disk drives for information storage includes at least one module with a plurality of drive bypass circuit boards, each including a drive bypass circuit board connector. At least one opening is provided between connectors to permit the flow of air between the connectors. Each drive bypass circuit board is a relatively flat circuit board with connectors on different edges of the board, wherein one of the connectors is the connector which receives the disk drive and the other connector connects to said drive bypass circuit board connector. The connectors, bypass circuit boards and drives are arranged such that when they are connected there is a path for air flow from outside the module alongside each bypass circuit board and its associated disk drive for cooling purposes without any backplane obstruction. Where the mass storage system is housed in an enclosure, at least one fan is mounted to force air from outside said enclosure through the spaces between said bypass boards and drives, preferably through the openings between the drive bypass circuit board connectors.
It will be appreciated that the present invention provides a novel high speed information processing and/or mass storage system particularly suitable for information and application servers. The system is scalable, fault tolerant, and reliable both because of its novel system architecture and its physical layout. Other features and advantages of the invention will become apparent from the following detailed description of exemplary and preferred embodiments when read in conjunction with the drawings which illustrate, by way of example, the principles of the invention.
One embodiment of an information or application server system in accordance with the present invention using the high speed information processing and mass storage system of the invention is illustrated in
The computer 102 preferably comprises single board high speed computer running a computer industry standard operating system software program such as, for example, Windows NT available from Microsoft Corporation. An operating system like Windows NT may be stripped down to remove those elements of the program which are not needed, if desired to preserve memory or to increase operating speed. Suitable conventional drivers of the type used for similar applications may be provided as necessary to support the particular architecture being implemented.
The computer may include a display such as a touch screen display, and various storage and peripheral devices (not shown) as required. The single board computer can include any of a wide number of suitable devices including, but not limited to, the Compact PCI CPU Board with Pentium Processor, Model No. ZT 5510, available from Ziatech Corporation. Modifications to enhance performance of the ZT 5510 can include an onboard 40 MB flash memory card for permanent storage of the non-reconfigurable portions of the Windows NT operating system software and an onboard, removable, PCMCIA 40 Mb flash memory card, “D2 FlashDisk” available from Sandisk Corporation for read/writeable storage of the reconfigurable portions of the Windows NT software.
The Fibre Channel controller 104 may be any suitable design according to the Fibre Channel Consortium created as a separate board or incorporated into the single board computer design. The communications interface or access card 106 may be any suitable device made in accordance with well known T-1 communications architecture, and/or architecture adapted for compatibility with other network and telecommunications architectures, protocols and topologies including, but not limited to, T-3, DS-3, OC-3 C, OC-12C, OC-192C, FDDI, SONET, SCSI, TCP/IP, HiPPI and ATM. In addition, the computers 102 and 110 may be networked together and with other computers through appropriate ethernet cards or other suitable networking techniques. The respective manufacturer, Fibre Channel Consortium and 120 Special Interest Group reference design data sheets and materials describing the detailed operating capabilities and specifications of each of the foregoing components are hereby incorporated by reference in their entirety. Also, further information concerning possible subsystems and connection protocols for the server are described in the referenced Dellacona applications.
Each module 202 has a module or chassis bypass circuit board 216 in the communication path of the Fibre Channel controller 104. The chassis bypass circuit board 216 of module 202A is provided with an optical input/output connector 218 for outputting electrical signals from the module 202A as light signals and for inputting light signals into the module 202A as electrical signals. Likewise, the modules 202B . . . 202 n have chassis bypass circuit boards 216 with associated input/output connectors 218 (not shown). As illustrated, the input/output connector 218 of the chassis bypass circuit board of module 202A is adapted to be connected via a light transmission medium such as optical fibers 220 to the optical input/output connector 218 (not shown) of the chassis bypass circuit board of module 202B.
As was previously noted, the controller 104 preferably is a conventional Fibre Channel Controller (FCC) which operates on a Fibre Channel protocol, and preferably is an arbitrated dual channel Fiber Channel Controller. The controller 104 provides a dual channel communication path within each module 202 between the computer 102 and each of the operable storage devices 212. As is described hereinafter in greater detail, the bypass circuit boards 210 ensure that the communication path is complete even if a storage device 212 is inoperable (i.e., is not operable at or above some minimum level as is hereinafter described in greater detail) or has been removed from the connector on the bypass circuit board.
In that regard, each of the storage devices 212 is preferably a high speed, high capacity, conventionally available disk drive which is removably connected to its associated bypass circuit board 210. Preferably, the disk drive plugs into a connector on the bypass circuit board 210 so that it can be readily removed and replaced or so that drives may be added to empty bypass circuit board positions, as needed to expand the storage capacity of the module. Each bypass circuit board 210 includes circuits which connect the controller 104 to the disk drive 212 when the disk drive is plugged in and is conveying to the bypass circuit board that it is operable at a certain minimum level. On the other hand, the bypass circuit board 210 connects the controller 104 directly to the next bypass circuit board, bypassing the disk drive 212, when the disk drive is not plugged in or is not operating at or above the minimum level. Any suitable, conventional disk drive of the type that runs self-diagnostics and provides a fault/no fault output signal may be used for this purpose.
With further reference to
With continued reference to
It will be appreciated that with the above described architecture, individual mass storage device modules 202 of the mass storage array 200 may be expanded internally by adding disk drives or other suitable storage devices, and bad disk drives may be replaced without affecting the operation of the rest of the module or the system it is used in. This provides an extremely versatile hot swappable, hot expandable, mass storage device array. In addition, when the demands of the system exceed the capacity of a single module, an additional module may be added, again without interrupting the operation of the rest of the array or its system.
As was explained above, the Fibre Channel controller provides a dual path through each module of the mass storage array 200. In accordance with the present invention, the system can be configured so that the dual path is a single path which traverses the mass storage array twice or two independent paths as is illustrated in
Referring now to
It will be appreciated by one skilled in the art that the
It will be appreciated that the system configuration illustrated in
It can be seen that system architecture according to the present invention lends itself to a wide variety of configurations to accommodate a variety of applications.
The A signal path from the backplane connector 230 is connected to a suitable conventional electronic switch 234. The B signal path from the backplane connector 230 likewise is connected to an electronic switch 236. The A and B signal paths from electronic switches 234 and 236 are connected to a bypass board storage card or drive connector 238 where they are routed to the storage device (e.g., a disk drive) 212.
The return A signal path from the bypass board drive connector 238 is connected to the switch 234, and the return B signal path from the connector 238 is connected to switch 236. A fault signal produced by the storage device to indicate its presence and its level of operability as was described above is applied to each of the electronic switches 234 and 236 to control the switching thereof. The A and B return paths from the switches 234 and 236 are connected to the bypass board backplane connector 230 where they are routed through the backplane 232 to the next bypass circuit board or to the Fibre Channel controller.
In operation, the A signal path enters the bypass circuit board and is connected to the switch 234. If the fault signal is not present (i.e., there is no fault and the signal is in a low or negative signal state) indicating that the storage device is not present or is inoperable, the switch 234 returns the A signal path to the bypass board backplane connector 230 thus bypassing the storage device 212. The B signal path similarly is looped back to the backplane connector 230 by the electronic switch 236 if the fault signal is low. On the other hand, the A and B signal paths are routed through the switches 234 and 236 to the storage device and then back through the switches when the fault signal is high or positive indicating the storage device is present and operable.
Referring now to
Typically, each of the modules 202 of a mass storage device array such as the one shown in
In accordance with one aspect of this physical structure of the invention, each of the drive bypass circuit boards 210 is a relatively thin circuit board. The circuit board is, however, unlike typical circuit boards built to receive a disk drive or other mass storage device. In such conventional circuit boards, the connector or plug which receives the plug-in disk drive typically is positioned so that the disk drive extends perpendicular to the plane of the board. For example, it is usual to have the bypass circuits and/or the communications paths between them on a backplane or midplane circuit board which extends across the module in a fashion similar to a computer motherboard which extends across the computer chassis and has connectors to receive various plug-in cards or boards. That arrangement creates an obstruction which makes it more difficult to effectively cool heat producing storage devices such as disk drives.
As, is illustrated in
It can be seen that between each connector 240 there is a space 244 through which air can readily be drawn or forced by a fan or other air circulation means as is necessary. Even if the structural members are part of a housing that surrounds the components, screening or other suitable openings can be provided so that the areas 244 permit sufficient air flow. It can also be seen that because the disk drive is plugged into the bypass circuit board so that their planes are parallel and not perpendicular, there is no obstruction of air flow.
It will also be appreciated that this arrangement is particularly suited for field service of the unit and is readily upgradeable. Connectors can be readily replaced in the field without the need to change a complete backplane or midplane board, and in some instances repairs of this sort can be carried out with little or no down time. In addition, the illustrated connection arrangement permits expansion without the limitations encountered when using a backplane or midplane with a fixed number of expansion slots and without the other physical and electrical limitations encountered with backplanes or midplanes.
Similar physical arrangements may be used to connect computers to their associated components to create the desired server configuration. For example, as illustrated in
Referring generally now to the block diagram of
The above-described exemplary embodiments are intended to be illustrative in all, respects, rather than restrictive, of the present invention. Thus the present invention is capable of many variations in detailed implementation. that can be derived from the description contained herein by a person skilled in the art. All such variations and modifications are considered to be within the scope and spirit of the present invention as defined by the following claims.