|Publication number||USRE36448 E|
|Application number||US 08/766,806|
|Publication date||Dec 14, 1999|
|Filing date||Dec 13, 1996|
|Priority date||Sep 13, 1991|
|Publication number||08766806, 766806, US RE36448 E, US RE36448E, US-E-RE36448, USRE36448 E, USRE36448E|
|Inventors||James T. Brady|
|Original Assignee||International Business Machines Corporation|
|Export Citation||BiBTeX, EndNote, RefMan|
|Patent Citations (7), Non-Patent Citations (2), Referenced by (7), Classifications (5), Legal Events (2)|
|External Links: USPTO, USPTO Assignment, Espacenet|
This application is a continuation of application Ser. No. 07/759,308 filed on Sep. 13, 1991 and now abandoned.
This invention relates to a method and means for generating parity for an array of redundant storage devices, such as disk files, and more particularly to means including a memory controller whereby (1) data transfers and hence bus bandwidth are reduced by minimizing movement of data to and from a dedicated parity generator that serves at least some of the storage devices of the array; (2) parity generation can be interspersed with data transfer; and (3) different calculations can be assigned to and/or shared between different ones of or portions of the memories associated with the parity generators.
As used herein in the specification and claims, "parity" is intended to connote parity as generated for use in redundant storage arrays, including exclusive OR (XOR) and other codes (e.g., the b-adjacent Reed-Solomon codes) that require only one data bit and the partial parity bit as modified on a data-block-by-data-block basis; and the parity generating code should use an associative and commutative calculation that is performed on a bit-by-bit basis within tile data blocks. Error Correction Code (ECC) is intended to connote use of conventional parity and ECC techniques for error detection and/or correction in memories. "Storage devices" is intended to include but not be limited to disk files, tape drives, optical disks, and cache memories "Data source" is intended to connote a channel interface, a memory, a storage device, a microprocessor on a bus, a bus-to-bus bridge, or any other device that can supply data to a bus.
The feasibility of redundant disk arrays is described by Patterson et al in "A case for redundant arrays of inexpensive disks", Report No. UCB/CSD 87/391, Computer Science Division, University of California/-Berkeley, December 1987.
The advent of small form-factored disks and various design improvements have made it feasible to use "parity" devices in arrays of external storage devices (such as disk arrays). Key to making a parity device usable is minimizing the performance effects and resources required to calculate parity. Parity is normally calculated in a special dedicated parity generator that is connected to the main data bus of the array controller. In addition to bus controls and logic to control the parity generator, multiple passes of the data are required on the bus. For high data rate arrays, such as one able to support the rated bandwidth of a High Performance Parallel Interface (HPPI) input/output bus, the HPPI interface can handle 200 MB/sec. However, a worst case condition (e.g. a normal write coupled with a read, with one device out of service), would require a bandwidth of 800 MB/sec if the array uses a conventional parity generator; or without the parity generator, a bandwidth of 400 MB/sec.
U.S. Pat. No. 3,876,978 discloses a system comprising an array of tape drives and means, including a specially dedicated one of these tape drives, for preventing catastrophic loss of data in one of the remaining tape drives. The dedicated tape drive, in conventional fashion, stores, as check bits, the XOR summation of the bits of the other tape drives at each respective bit position. More specifically, information read from a tape is sent to a buffer, fed to a two-way XOR, buffered, and then written back on the tape. However, this system merely reads and writes (calculates) parity, and does not and cannot read and write data. It constitutes a dedicated resource that only generates parity for a plurality of tape drives. Moreover, this arrangement requires numerous data transfers and a parity calculation sequence that is fixed.
U.S. Pat. No. 3,976,982 discloses an apparatus for manipulating graphic images by reading a base image stored at one location in an accumulating memory, combining and XORing it with a sequence read from another raster source, and then writing the combined image in said memory at the same location. However, there is no teaching or suggestion that this approach be modified and used to create a new method for parity generation.
There is a need for an improved method and means for calculating and generating parity for external storage devices, such as disk arrays, which (1) permits a reduction in the bandwidth of the array controller bus and the number of data transfers required; (2) permits transfer of the parity blocks in any order and interspersed with other parity calculations; (3) permits each storage area in the memory to be used for data or parity, thereby simplifying system design, and (4) in addition to controlling the normal operations of the memory, responds to a special "write parity" signal to write parity.
Parity encoding is performed on a plurality of data strings moving between a memory (23) connected via a read path (31) to an interconnection means (24) operatively connected to a corresponding plurality of storage devices. Parity for the storage devices is calculated using a parity generator (34) connecting a write buffer means (35) with the memory via a write path (30). Circuitry responsive (36,37) to control signals (38) from the interconnection means conveys data representing a then current partial parity from the memory via the read path to the parity generator. The then current partial parity is logically and recursively combined with data from the write buffer means to update the calculation of partial parity until, after all partial parity calculations are completed, final parity calculation is completed.
This technique reduces the number of data transfers and bandwidth required to effectuate the parity computations over the string. The data strings or portions thereof may be logically combined in any order presented to said buffer means. For the parity encoding, a code is used that is both associative and commutative.
Error detection and correction is preferably provided for the data in the memory by interposing an ECC generator (32) between the write path and the memory and interposing an ECC check/correction unit (33) between the read path and a control device responsive to the control signals.
FIG. 1 is a schematic drawing of a redundant disk array with a dedicated parity generator, as taught in the prior art;
FIG. 2 is a redundant disk array including a memory controller embodying the invention; and
FIG. 3 is a more detailed view of the memory controller depicted in FIG. 2.
The redundant disk array illustrated in FIG. 1 corresponds to one heretofore proposed and known as UC Berkeley's RAID II design. This array comprises a channel interface 10, a parity generator 11, and a memory 12, each connected to a data bus 13. Memory 12 comprises a memory controller 14 and a memory array 15.
Assume that n blocks of data are streamed via channel interface 10 to bus 13 and, via port a, are stored serially in memory 12. In response to a control signal via bus 13 to port c, the n blocks of data are sent serially from memory 12, via port a, bus 13 and port d to generator 11. Generator 11 calculates parity by XORing the n blocks. Then, in response to a control signal via bus 13 to port f, generator 11 transmits the parity, as calculated, via ports e and b back to memory 12. From memory 12 the parity is transmitted via a SCSI (Small Computer System Interface) controller 16 to the appropriate one 17' of a plurality of disk files 17.
This parity generating technique thus results in 2n+2 transfers along the controller data bus 13.
As illustrated in FIG. 2, the redundant disk array embodying the invention comprises a channel interface 20 and a plurality of identical memories 21, such as 21a, 21b. Each memory 21 comprises a memory array 23 (which may be identical with memory array 12) and also, comprises, according to a feature of the invention, a memory controller 22 that includes a parity generator.
Assume that the redundant disk array includes n+1 disk files 25, and that it is desired to write n data blocks on n different disk files, calculate parity, and write the parity, as calculated for the said n blocks, on the n+1th disk file. As disclosed herein, the n data blocks, each k*m bits wide, are supplied via channel interface 20 to an interconnection means (herein illustrated as a data bus 24) and directed serially, in bursts of m bits, to input ports g of each memory 21. One of the memories 21 (herein assumed as 21a) will have been preselected to calculate parity for this particular data transfer; and that designated memory 21a would respond to a control signal from bus 24, via its port i, to perform an XOR to the contents of a portion or location P of the memory array 23 allocated for parity calculation. That allocated location P either will have been preinitialized to zero to enable the XOR to occur, or the first block to arrive will be stored with a conventional store, precluding the need for preinitialization. As the data blocks continue to arrive via channel interface 20, they will be XOR'd one-by-one to the partial parity as calculated at said allocated location P. When the last of the n blocks is transmitted, parity will immediately be available in memory 21a and be written in memory array 23 and also via disk controller 26 to the appropriate disk file 25'.
Meanwhile, as illustrated, the n data blocks are stored in location D of the array 23 of memory 21b in response to control signals via port(s) i specifying normal storage. Once parity is calculated in memory 21a, it will be sent, in response to a control signal supplied from bus 24 via port i, onto the bus via port h and gated out to disk controller 26. Controller 26 then directs the parity to the appropriate disk 25'.
FIG. 3 depicts, in greater detail, how each memory 21 is configured to implement the invention. As illustrated, for simplification of showing and description and to illustrate a variation of the invention, it will be assumed that the parity location P and data location(s) D will be in the same memory 21 (although in practice and from preceding description it will be understood this is not required).
According to the invention, the memory controller 22 preferably comprises an ECC generator 32 and, an ECC check and correction unit 33 and, in all events, comprises a disk parity generator 34, a write buffer 35, a control device 36, and a switch 37. Write path 30 connects ECC generator 32 to memory array 23, and a read path 31 connects the memory array to the ECC check/correction unit 33.
In operation, to write data into memory in normal write mode, a control signal is transmitted from data bus 24 via line 38 to control device 36. This control signal causes device 36, via line 39, to set disk parity generator 34 to a pass-through mode, and also, via line 47, to condition ECC generator 32. This gates data from data bus 24 via line 42, write buffer 35, disk parity generator 34, ECC, generator 32 and write path 30 to the location D in memory array 23 of memory 21 preselected by control device 36 via line 44. After the data is thus transmitted, control device 36 will condition itself to a wait state and await the next instruction.
To read data in normal read mode, a control signal from bus 24 via line 38 conditions control device 36 to select via line 44 the location D in the memory array 23 of memory 21 that is to be accessed. The control signal from line 38 also causes device 36 to condition ECC check/correction unit 33 via line 48 and to activate switch 37 to connect line 40 to line 41. As a result, the data to be read is gated from location D in memory array 23 via read path 31 to the ECC check/correction unit 33. Unit 33 checks to see if ECC parity is correct and, if it is not, initiates correction of the data, in conventional manner, a function not here described because not pertinent to the present invention. Assuming that the data is correct, it will then be transmitted from location D via switch 37 (as conditioned by device 36) from line 40 via line 41.
According to the invention, the memory controller 21 has a "write parity" mode in which all or any designated portion of memory array 23 may be used for parity calculation, thereby eliminating the need, as previously required, to transfer data to and from a dedicated parity calculation unit. In this mode, data is written from location D to an allocated memory address or location P in memory array 23 and logically combined with the existing contents of said memory location to update the parity calculation. This combining step is recursively repeated until the final parity calculation is generated.
With the embodiment illustrated in FIG. 3, wherein locations D and P are in the same memory array 23, two sets of control signals are transmitted sequentially via line 38 to control device 36. One set of control signals identifies the location P at which parity calculation is to be updated and that an XOR write operation is to be performed. The other set of control signals identifies the location D from which data is to be read into the write buffer 35.
More specifically, switch 37 is conditioned by the one set of signals to connect line 40 to line 43, and by the other set of signals to connect line 40 to line 45. With lines 40,43 connected, the contents of location P are gated via unit 33 to write buffer 35. With lines 40, 45 connected, the contents of location D are gated via unit 33 to write buffer 35. The data written into write buffer 35 from locations P and D is synchronized. When the nth bytes from location P line up with the data from location D, the synchronized m-bit data words will be moved in parallel from write buffer 35 over two paths into disk parity generator 34. Generator 34 will have been conditioned by control device 36 via line 39 to XOR the two m-bit data words together bit-by-bit, then transmit the XOR'd m-bit data word to ECC generator 32, which will have been conditioned via line 47 to calculate ECC parity for the memory array 23. Once parity is calculated, the data and memory array parity are transmitted via write path 30 to memory array 23 and there stored in location P. This process is repeated recursively until the required number of words from locations D and P in memory array 23 have been XOR'd together to calculate partial parity for one block. This process is then repeated recursively for successive blocks, updating the parity calculation stored at location P until the final parity calculation is completed. Memory 21 resumes a wait state awaiting the next instruction after each partial parity calculation.
It should be noted that ECC generator 32 and ECC check/correction unit 33 preferably are included to provide error detection and correction of the data in the associated memory array 23 in the event there is a drop-out of a bit in the corresponding memory 21. However, the invention may be implemented even if ECC generator 32 and ECC check/correction unit 33 are eliminated. In such case, disk parity generator 34 would be connected directly to write path 30; read path 31 would be connected directly to line 40; and lines 47 and 48 would be eliminated. Accordingly, in interpreting the claims, applicant does not wish to be limited to configurations requiring inclusion of generator 32 and unit 33.
Parity calculation can be implemented in memory array 23 either on chip, on card or off card; however, the closer the logic is to the chip the better the opportunity to reduce the cost of bandwidth.
It will now be apparent that the number of data transfers and hence the bandwidth of the data bus or other interconnection means can be minimized because the dedicated parity unit of the prior art is eliminated and replaced with a small amount of logic in the memory controller 22. Also, microcode otherwise needed to control a conventional dedicated parity unit may be eliminated, thus providing cost savings that increase according to the amount of buffering that had been required for said dedicated parity unit. Each storage block or area in the memory array 23 can be used as a buffer or cache memory for storing either data or parity; and design flexibility is increased because data from any data source can be stored or XOR'd to any storage block. Transfer of data blocks during parity generation can be implemented in any order, and the transfer can be interspersed with other parity calculations. Finally, if implemented with a multidestination bus, the invention permits storing of both data and parity information to be accomplished with a single bus transfer. Moreover, applicant's parity generation technology results in only n+1 transfers along controller bus 24.
It should also be noted that one memory controller may control all parity and ECC calculation for one or more external storage devices. Alternatively, parity and ECC calculation may be shared or allocated between a plurality of memory controllers, with each controller calculating a portion of the total parity or ECC.
Finally, it should be noted that the data locations, such as D, and the parity location P may be in the memory arrays 23 of different memory controllers (e.g. 21a and 21b, as depicted in FIG. 2); or if preferred, the locations D and P may be in the same memory array 23 (as depicted in FIG. 3). If locations D and P are in the same memory array, data may be moved from location D to disk parity generator 34 without requiring transmission via data bus 24. Diode 46 is interposed in line 42 to isolate line 45 from the data bus to permit data to be supplied from a location D in the same memory array or via the; data bus if the location D is in a different memory array.
While the invention has been shown and described with respect to a preferred embodiment thereof, it will be understood by those skilled in the art that changes in form and detail may be made in this embodiment without departing from the scope and teaching of the invention. Accordingly, the memory controller and method herein disclosed are to be considered merely as illustrative, and the invention is to be limited only as specified in the claims.
|Cited Patent||Filing date||Publication date||Applicant||Title|
|US3876978 *||Jun 4, 1973||Apr 8, 1975||Ibm||Archival data protection|
|US3976982 *||May 12, 1975||Aug 24, 1976||International Business Machines Corporation||Apparatus for image manipulation|
|US4627059 *||Jul 19, 1984||Dec 2, 1986||Siemens Aktiengesellschaft||Circuit arrangement for telecommunications systems, particularly telephone switching systems, having data protection by way of parity bits|
|US4916704 *||Sep 4, 1987||Apr 10, 1990||Digital Equipment Corporation||Interface of non-fault tolerant components to fault tolerant system|
|US5142541 *||Dec 30, 1988||Aug 25, 1992||Samsung Electronics Co., Ltd.||Error-bit generating circuit for use in a non-volatile semiconductor memory device|
|US5233616 *||Oct 1, 1990||Aug 3, 1993||Digital Equipment Corporation||Write-back cache with ECC protection|
|US5515381 *||Jan 14, 1992||May 7, 1996||Tandem Computers Incorporated||Sequential parity correction for error-correcting RAM array|
|1||"A Case for Redundant Arrays of Inexpensive Disk (RAID)", D.A. Patterson, G. Gibson & R. Katz, No. UCB/CSD 87/391, Dec., 1987, Univ. of Berkeley.|
|2||*||A Case for Redundant Arrays of Inexpensive Disk (RAID) , D.A. Patterson, G. Gibson & R. Katz, No. UCB/CSD 87/391, Dec., 1987, Univ. of Berkeley.|
|Citing Patent||Filing date||Publication date||Applicant||Title|
|US6681340||Feb 23, 2001||Jan 20, 2004||International Business Machines Corporation||Efficient implementation of error correction code scheme|
|US6986097 *||Feb 21, 2003||Jan 10, 2006||Ciena Corporation||Method and apparatus for generating parity bits in a forward error correction (FEC) system|
|US8566672||Mar 22, 2011||Oct 22, 2013||Freescale Semiconductor, Inc.||Selective checkbit modification for error correction|
|US8607121||Apr 29, 2011||Dec 10, 2013||Freescale Semiconductor, Inc.||Selective error detection and error correction for a memory interface|
|US8990657||Jun 14, 2011||Mar 24, 2015||Freescale Semiconductor, Inc.||Selective masking for error correction|
|US8990660||Sep 13, 2010||Mar 24, 2015||Freescale Semiconductor, Inc.||Data processing system having end-to-end error correction and method therefor|
|US20070144244 *||Dec 20, 2006||Jun 28, 2007||Karma Technology, Inc.||Probe module with integrated actuator for a probe microscope|
|U.S. Classification||714/763, 714/48|
|Jan 7, 2002||FPAY||Fee payment|
Year of fee payment: 8
|Jan 9, 2006||FPAY||Fee payment|
Year of fee payment: 12