US20050243611A1 - Flexible data replication mechanism - Google Patents

Flexible data replication mechanism Download PDF

Info

Publication number
US20050243611A1
US20050243611A1 US11/151,499 US15149905A US2005243611A1 US 20050243611 A1 US20050243611 A1 US 20050243611A1 US 15149905 A US15149905 A US 15149905A US 2005243611 A1 US2005243611 A1 US 2005243611A1
Authority
US
United States
Prior art keywords
data
storage
controller
port
data replication
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US11/151,499
Inventor
Clark Lubbers
Susan Elkington
Randy Hess
Stephen Sicola
James McCarty
Anuja Korgaonkar
Jason Leveille
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Individual
Original Assignee
Individual
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Individual filed Critical Individual
Priority to US11/151,499 priority Critical patent/US20050243611A1/en
Publication of US20050243611A1 publication Critical patent/US20050243611A1/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/16Error detection or correction of the data by redundancy in hardware
    • G06F11/20Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements
    • G06F11/2002Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements where interconnections or communication control functionality are redundant
    • G06F11/2007Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements where interconnections or communication control functionality are redundant using redundant communication media
    • G06F11/201Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements where interconnections or communication control functionality are redundant using redundant communication media between storage system components
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/1001Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
    • H04L67/1004Server selection for load balancing
    • H04L67/1008Server selection for load balancing based on parameters of servers, e.g. available memory or workload
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/1001Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
    • H04L67/1004Server selection for load balancing
    • H04L67/101Server selection for load balancing based on network conditions
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/1001Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
    • H04L67/1034Reaction to server failures by a load balancer
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/1097Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L9/00Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
    • H04L9/40Network security protocols
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/1001Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/1001Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
    • H04L67/10015Access to distributed or replicated servers, e.g. using brokers
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/40Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass for recovering from a failure of a protocol instance or entity, e.g. service redundancy protocols, protocol state redundancy or protocol service redirection
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10TECHNICAL SUBJECTS COVERED BY FORMER USPC
    • Y10STECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10S707/00Data processing: database and file management or data structures
    • Y10S707/99951File or database maintenance
    • Y10S707/99952Coherency, e.g. same view to multiple users
    • Y10S707/99953Recoverability
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10TECHNICAL SUBJECTS COVERED BY FORMER USPC
    • Y10STECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10S707/00Data processing: database and file management or data structures
    • Y10S707/99951File or database maintenance
    • Y10S707/99952Coherency, e.g. same view to multiple users
    • Y10S707/99955Archiving or backup

Definitions

  • the present invention relates generally to error recovery in data storage systems, and more specifically, to a system for providing controller-based remote data replication using a redundantly configured Fibre Channel Storage Area Network to support data recovery after an error event and enhance data distribution and migration.
  • DAS direct-attached storage
  • NAS network-attached storage
  • SAN storage area network
  • Mass data storage systems are implemented in networks or fabrics that provide means for communicating data between systems that use data, and the storage systems that implement the physical storage.
  • host computers act as storage servers and are coupled to the network and configured with several disk drives that cumulatively provide more storage capacity or different storage functions (e.g., data protection) than could be implemented by a DAS system.
  • a server dedicated to data storage can provide various degrees of redundancy and mirroring to improve access performance, availability and reliability of stored data.
  • Collecting storage sub-systems, where a separate server manages each sub-system, can form a large storage system.
  • virtualized storage systems such as the StorageWorks® Enterprise Virtual Array announced by Compaq Corporation in October, 2001 provide storage controllers within a fabric or network that present virtualized storage to hosts that require data storage in a manner that enables the host to be uninvolved in the physical configuration, allocation and management of the storage devices.
  • StorageWorks is a registered trademark of Compaq Computer Corporation. In this system, hosts simply access logical units of storage that appear to the host as a range of logical address space. Virtualization improves performance and utilization of storage.
  • SAN systems enable the possibility of storing multiple copies or “replicas” of data at various physical locations throughout the system.
  • Data replication across multiple sites is desirable for a variety of reasons.
  • copies of data stored at different physical locations is desired.
  • a replica located at an alternate site can allow access to the data.
  • Replicated data can also theoretically improve access in normal operation in that replicas can be accessed in parallel, avoiding bottlenecks associated with accessing a single copy of data from multiple systems.
  • prior systems were organized such that one site had a primary role and another site was a replica. Access requests were handled by the primary site until failure, at which time the replica became active.
  • the replica provided little benefit until failure.
  • the resources allocated to creating and managing replicas provided minimal load balancing benefit that would enable data access requests to be directed intelligently to replicas such that resources were used more efficiently.
  • network delays associated with accessing a topologically remote storage subsystem could be lessened.
  • disaster tolerance In a data storage environment, disaster tolerance requirements include providing for replicated data and redundant storage to support recovery after the event. In order to provide a safe physical distance between the original data and the data to be backed up, the data is migrated from one storage subsystem or physical site to another subsystem or site. It is also desirable for user applications to continue to run while data replication proceeds in the background. Data warehousing, ‘continuous computing’, and enterprise applications all benefit from remote copy capabilities.
  • Compaq Corporation introduced a data replication management product in its Array Controller Software (ACS) operating on an HSG80 storage controller and described in U.S. patent application Ser. No. 09/539,745 assigned to the assignee of the present application and incorporated herein by reference.
  • This system implemented architecture with redundant storage controllers at each site. Two sites could be paired to enable data replication.
  • the HSG80 architecture defined relatively constrained roles for the components, which resulted in inflexibility.
  • each of the controllers comprised one port that was dedicated to user data, and a separate port that was dedicated to data replication functions. Even where redundant fabrics were implemented, for a given controller both of these ports were coupled to a common fabric switch. Despite the fact that each controller had two ports for communicating with other controllers, one of the ports was constrained in the role of handling user data, and the other port was constrained in the role of handling data replication. Failure of either port would be, in effect, a failure of the entire controller and force migration of storage managed by the failed controller to the redundant controller. Similarly, failure of a communication link or fabric coupled to one port or the other would render the controller unable to perform its tasks and force migration to the redundant controller. Such migration was disruptive and typically required manual intervention and time in which data was unavailable.
  • prior data replication management solutions simplified the implementation issues by assigning fixed roles to storage locations.
  • a particular storage site would be designated as a primary when it handled operational data traffic, and another site would be designated as a secondary or backup site.
  • Such architectures were unidirectional in that the backup site was not available for operational data transactions until the failure of the primary site.
  • Such rigidly assigned roles limited the ability to share storage resources across multiple topologically distributed hosts.
  • configuration of such systems was complex as it was necessary to access and program storage controllers at both the primary and secondary sites specifically for their designated roles. This complexity made it impractical to expand data replication to more than two sites.
  • a data storage system capable of providing flexible data replication services without the direct involvement of the host computer.
  • a data storage system is needed that is readily extensible to provide multiple replication, load balancing, and disaster tolerance without limitations imposed by designating rigid roles for the system components.
  • the present invention involves architecture for flexible data replication management comprising first and second storage cells at separate topological locations on a storage area network.
  • Each storage location comprises a storage controller having first and second ports coupled to a communication network, and a pool of storage devices accessible through the controller.
  • Each of the first and second ports communicates both user data and replication data with the communication network.
  • each storage location is enabled to operate as a source that initiates replication of data stored within its pool of storage devices, and as a destination for storing replicas of data stored within storage devices of another storage location.
  • the communication network comprises redundant sub-networks, and each controller is coupled such that the first port couples to one of the sub-networks and the second port couples to another of the sub-networks.
  • FIG. 1 shows a storage area network (SAN) environment in which the present invention is implemented
  • FIG. 2 illustrates an alternative SAN environment embodying the present invention
  • FIG. 3 shows an implementation emphasizing redundant connections to improve disaster tolerance
  • FIG. 4 illustrates an implementation with less redundancy than the implementation of FIG. 3 ;
  • FIG. 5 illustrates a basic implementation of the present invention
  • FIG. 6 shows data flow relationships in a data replication management operation in accordance with the present invention.
  • FIG. 7 illustrates logical relationships between data structures of the present invention.
  • the present invention describes a data replication management (DRM) architecture comprising a plurality of storage cells interconnected by a data communication network such as a fibre channel fabric.
  • DRM data replication management
  • the present invention emphasizes symmetry and peer-cooperative relationships amongst the elements to provide greater flexibility in configuration and operation.
  • Storage cells are for the most part autonomous units of virtualized storage that implement hundreds of gigabytes to terabytes of storage capacity.
  • each storage cell can act as a source or primary location for data storage, and at the same time act as a destination or secondary location holding a replica data from a primary location of another storage cell.
  • the architecture of the present invention seeks to minimize rigid roles placed on interface ports and connections such that all available connections between components can support both host data traffic (e.g., traffic involving data that is to be stored and retrieved), and DRM traffic (e.g., traffic related to managing replicas between storage cells). In this manner, resources are efficiently deployed and operational performance is improved.
  • host data traffic e.g., traffic involving data that is to be stored and retrieved
  • DRM traffic e.g., traffic related to managing replicas between storage cells
  • network connectivity is illustrated by fibre channel mechanisms, however, other network mechanisms provide suitable functionality in particular environments.
  • storage capacity is presented as SCSI (small computer system interface) logical units, although the particular protocol is readily varied to meet the needs of a particular application.
  • FIG. 1 illustrates a multi-site storage area network (SAN) implementing features in accordance with present invention.
  • the example shown in FIG. 1 includes only three sites (Site 1 , Site 2 and Site 3 ) for to ease illustration and understanding, however, it should be understood that any number of sites may be provided to meet the needs of a particular application.
  • the various sites may be physically proximate, or separated by an arbitrary distance.
  • a particular implementation may well limit the number of possible sites, and may well limit the maximum or minimum physical separation of sites.
  • Each site includes one or more storage cells 101 , such as cells 101 A, 101 B, 101 C and 101 D. Any number of storage cells 101 maybe included in any site, although the number implemented in any particular application may be constrained to meet the needs of that application.
  • host computer 102 such as host computers 102 A, 102 B and 102 C.
  • Host computers are generally machines that consume or require data storage.
  • Typical host computers 102 demand large quantities of storage such as mainframes, web servers, transaction processors, and the like.
  • a host computer 102 may comprise a computer of any processing capacity that requires or benefits from network storage either to reduce cost of the host 102 , implement more storage capacity than practical in a host 102 , share data amongst multiple hosts 102 , or the like.
  • a host computer 102 may couple to the storage cells 101 via a connection to network 103 such as illustrated by host computers 102 A and 102 B.
  • a host computer 102 will be located in a site such as host computer 102 C located in site 2 .
  • the topological location of host computers 102 is a matter of design choice selected to meet the needs of the particular application.
  • one or more host computers 102 will be located at a site. It should be appreciated, however, that host computers 102 have little to do with the management and configuration of the DRM system of the present invention, other than the reality that as consumers of data, the DRM system is preferably able to satisfy storage needs of host computers 102 at various physical and topological locations.
  • Storage cells 101 and hosts 102 couple to data communication network 103 .
  • Storage cells 101 implement a quantity of data storage capacity that is accessible through storage controllers 105 that implement one or more connections to network 101 .
  • Storage cells 101 typically implement hundreds of gigabytes to terabytes of physical storage capacity.
  • controllers 105 virtualize the physical storage capacity such that it is configurable into logical units (LUNs) of storage capacity.
  • the LUNs implement an arbitrary quantity of logical address block storage, where each LUN has a specified level of data protection such as RAID 0-5 data protection.
  • the logical construction or context of a storage cell 101 comprises the logical and data structures that are used to represent, manage and configure the raw physical storage capacity into devices that can be accessed by hosts 102 in a desired manner.
  • controllers 105 implement and manage various logical objects that implement data structures and behavior to represent various components of each storage cell 101 , preferably without involvement if hosts 102 . All objects in the system are identified with a universally unique identification (UUID).
  • UUIDs are stored persistently in metadata structures within the physical storage which enables all or part of a storage cell, and all the LUNs implemented therein, to be mapped and presented by any storage controller 105 .
  • the ability to present the LUNs and the context in which they exist is useful, as described in greater detail below, in event of a controller failure at one of the replica sites.
  • LUNs are also identified by a worldwide LUN ID (WWLID) which identifies the LUN to hosts 102 .
  • WWLID worldwide LUN ID
  • Hosts 102 access physical storage capacity by addressing read and write operations to specified LUNs using the WWLID, and can be otherwise unaware of the physical storage architecture or data protection strategy for a particular LUN that is being accessed.
  • Storage controllers 105 manage the tasks of allocating physical storage capacity to specified LUNs, monitoring and maintaining integrity of the LUNs, moving data between physical storage devices, and other functions that maintain integrity and availability of the data stored therein.
  • Network 103 comprises any of a variety of available networks, and may comprises a plurality of interconnected networks.
  • network 103 comprises at least two independent fibre channel fabrics to provide redundancy.
  • These fibre channel fabrics may comprise long-distance connection mechanism 201 such as asynchronous transfer mode (ATM) and Internet protocol (IP) connections that enable sites to be separated by arbitrary distances.
  • ATM asynchronous transfer mode
  • IP Internet protocol
  • At least one SAN management appliance (SMA) 107 is coupled to network 103 to enable connections to storage cells 101 .
  • SMA SAN management appliance
  • a number of SMAs 107 are provided, and typically an SMA 107 is located at each site to provide management operation for the storage cells 101 at that site.
  • each SMA 107 communicates through network 103 , the physical location of SMAs 107 is arbitrary.
  • SMAs 107 are preferably implemented at topological locations that provide redundant connections to the storage cells 101 .
  • Network 103 is accessible directly or indirectly to all components at Site 1 , Site 2 , and Site 3 including hosts 102 and controllers 105 .
  • each component has redundant links to network 103
  • network 103 is preferably implemented as having redundant sub-networks as shown in FIG. 2 .
  • Redundancy provides connectivity in event of failure or degradation of some portions of network 103 .
  • Redundancy also enables connectivity in event of failure or degradation of controllers 105 and/or interface components of hosts 102 .
  • each controller might have two host ports to a network, however, one of the ports was designed to support operational data traffic while the other port was dedicated to data replication operations.
  • storage cell 101 is able to function as a primary storage location for any of hosts 102 , and at the same time function as a secondary or alternative storage location for a replica of data from another storage cell 101 .
  • This feature is referred to as “bi-directionality”.
  • the present invention provides for flexible assignment at a much finer level of granularity, preferably such that individual virtual disks can be designated as either primary or alternate.
  • One effect of such flexibility is that it becomes practical to implement enormous quantities of storage in each storage cell 101 as the capacity at each site can be readily applied to various applications with disparate storage needs.
  • FIG. 3 illustrates various features of the present invention embodied in a disaster tolerant configuration.
  • Storage management appliances 107 include a Bridge and a data replication management console (DRMC) which are cooperative software processes that each present a user interface for configuration and monitoring of DRM processes.
  • the Bridge functions as an element manager for controllers 105 , and handles basic functions of connecting to the communication network 103 , issuing commands to controllers 105 , and implementing security policies. All customer configuration and monitoring of DRM activities goes through the Bridge.
  • the DRMC is provides increased functionality over the Bridge interface, by managing solely DRM storage and providing assists, such as with site recovery.
  • One goal of a disaster tolerant configuration in FIG. 3 is to provide hardware redundancy for multiple levels of fault tolerance in order to keep the primary and alternate sites running, almost at ‘non-stop’ metrics (seconds per year downtime, barring disaster or planned downtime).
  • DRM will failover to a redundant component at that site to allow continued operation. If a significant failure happens to the primary site, then data processing can be resumed at the alternate site where the data is intact. Database or application recovery is achieved with little or no downtime.
  • redundant components are provided, they are preferably interchangeable in accordance with the present invention such that functionality of one component can be taken over by a redundant component.
  • node ports in the SMA 107 are redundant such that failure of one node port can be handled by moving functions handled by that port over to the other, non-failed node-port.
  • Physical storage devices 302 are coupled to controllers 105 by a high-speed redundant connection such as fibre channel arbitrated loop (FCAL) connections 303 in FIG. 3 .
  • FCAL fibre channel arbitrated loop
  • An FCAL connection allows each of tens or hundreds of individual hard disk drives to be accessed not only at high speed, but by either controller 105 in a storage cell. Further, by providing redundant FCALs, one FCAL can fail while each storage device 302 remains accessible by either controller.
  • the present invention is largely independent of the specific implementation of physical storage devices 302 , and it is contemplated that other physical storage architectures will provide suitable equivalents for purposes of the present invention.
  • FIG. 3 provides for high availability with a dual fabric implemented by switches 301 , dual hosts 102 , and dual storage topology, where a single switch, host, or storage can fail and the system can still continue access on the SAN.
  • Each fabric is composed of two switches 301 , with the switches connected to each other over what is called an E-port, or expansion port.
  • the E-port can be a connection with distances that vary depending on technology. Fibre Channel allows for 10 km and with extenders up to 100 km. ATM provides for ‘round the world’ distances. New technology, called FC-BB (Fibre Channel Backbone), provides the opportunity to extend Fibre Channel over leased Telco lines (also called WAN tunneling).
  • FC-BB Fibre Channel Backbone
  • Hosts 102 preferably run multi-pathing software that dynamically allows failover between storage paths as well as static load balancing of storage volumes (LUNs) between the paths of controllers 102 .
  • Multi-pathing software enables a host 102 to identify multiple connection options that lead to desired storage, and select amongst the available paths based on selected criteria. These criteria may vary widely, but typically include path availability, path performance, path load, controller load, and the like. This allows for applications to continue given any failure of fabric or the controllers.
  • a lower cost variant of this configuration uses a fabric comprising only one switch, such that there is no E-port. Such a configuration may be appropriate for shorter distances.
  • FIG. 4 illustrates a configuration of the present invention that emphasizes data movement rather than redundancy of the example in FIG. 3 .
  • the configuration of FIG. 4 provides a lower cost solution ideal for customers only interested in data backup, data migration, data distribution, and data mining.
  • FIG. 4 has a single fabric formed by two interconnected switches. With shorter distances, an even lower cost variant could use only one switch, such that there is no E-port.
  • FIG. 5 illustrates an even more basic configuration using a fabric having a single switch 301 , one host 102 , and non-redundant controllers 105 .
  • ports of controllers 105 are configured to handle both operational data transactions and DRM traffic, a single port connection between each controller 105 and switch 301 is all that is necessary to implement data replication features of the present invention.
  • the basic configuration shown in FIG. 5 sacrifices much redundancy, and would not be recommended for most applications. It is important to understand that a great deal of flexibility exists in configuring the present invention for specific applications. Redundant connections and components can be added and removed flexibly to meet the needs of a particular application. Hence, various features of the configurations shown in FIG. 1 through FIG. 5 can be mixed and matched to configure a specific implementation.
  • a SAN such as shown in FIG. 1 through FIG. 5 , is established by coupling controllers 105 and hosts 102 to switch 301 , or a fabric formed from multiple switches 301 .
  • Each of the devices performs a log in operation according to applicable fibre channel standards. The log in operation involves an exchange of information in which each device is registered with the fabric 103 , and devices can discover other devices that are registered with fabric 103 .
  • each controller 105 is identified by a storage cell unique identification number (UUID) which is stored by a name server in fabric 103 . By way of this registration and discovery process, each controller 105 can obtain knowledge necessary to connect to any other controller 105 .
  • UUID storage cell unique identification number
  • the source controller 105 can readily identify each available alternate controller 105 and continue operation without loss of data or requiring manual intervention.
  • a connection failure or link failure is distinct from a controller failure in that the controllers 105 remain operational, but unable to maintain a data communication link with one or more other controllers 105 .
  • the source controller has several options, depending on the nature of the failure. When the source controller 105 is unable to link to fabric 103 , suggesting a failure in a port of switch 301 , the controller 105 can attempt to use an alternate link to fabric 103 , or a redundant fabric 103 if available.
  • the source controller 105 attempts to establish a new link with a redundant port of the destination controller 105 .
  • there will be no need to reconstruct the storage cell 101 and data replication operations will proceed with substantially no interruption, and in most cases without any need to notify the host 102 of the aberrant condition.
  • prior systems had only one port on each controller available for DRM operations and so a link failure would require reconstruction of either the source or destination LUN in an alternate controller.
  • a link failure may require migration of a LUN from one controller 105 to another. For example, if all ports on either the destination or source 5 controllers 105 failed (which would typically be considered a controller failure), or several ports to fabric 103 failed simultaneously in particular combination, recovery would require migrating presentation of one or more LUNs from one controller 105 to a redundant or partner controller 105 in a manner similar to a controller failure event.
  • This migration can be performed efficiently using the mechanisms for mapping and presenting a storage cell 101 described above. Notably, all LUNs do not need to be presented before pending data replication operations can be completed. In implementations where the LUNs are virtualized, as described in co-pending patent application Ser. No.
  • FIG. 6 shows hierarchical relationships defined in accordance with the present invention to model data replication management.
  • a storage cell 101 Prior to performing DRM operations, a storage cell 101 is implemented at each site, and a virtual disk 601 is allocated within a storage cell 101 that is handing operational data with one or more hosts 102 .
  • one virtual disk 601 is designated as a source, although it should be understood that a source virtual disk 601 might be allocated at any site implementing the DRM features of the present invention.
  • Virtual disk 601 may comprise a plurality of physical storage resources that span multiple physical drives within the storage cell 101 , and may implement any desired capacity and data protection type.
  • a destination virtual disk 602 is allocated within a storage cell 101 at a designated alternative or destination site. In normal operation, write operations are directed to source virtual disk 601 , and copied in a background process to one or more destination virtual disks 602 . Destination virtual disk has the same logical storage capacity as the source virtual disk, but may provide a different data protection configuration. Controllers 105 of the destination storage cell handle the allocation of destination virtual disk 602 autonomously. This allocation involves creating data structures that map logical addresses to physical storage capacity, and in a particular implementation involve processed described in greater detail in U.S. patent application Ser. No. 10/040,194.
  • the actual allocation and copying of data may take up to several minutes to several hours in the case of storage of many gigabytes or terabytes.
  • These processes can be streamlined by allocating and copying only logical storage blocks that are used in source virtual disk 601 .
  • a virtual disk 601 may implement 1 terabyte of storage, but a sparsely populated example may use only a few gigabytes of capacity to store actual data.
  • destination virtual disk 602 will reserve resources needed to implement the entire 1 terabyte, but will only allocate and copy the locations that are actually used to store data and may allocate the other locations. This greatly reduces the time required to create replicates.
  • a “copy set” is a set of member virtual disks where each member virtual disk is a replica of the others, and may comprise any number of replica virtual disks. While it may be easier to describe and understand the present invention by designating one virtual disk as a source or original virtual disk and the others as destinations or replicas, it is important to understand that once the copy set is created each member virtual disk is essentially a peer of all others.
  • To create a copy set the user designates a virtual disk as the source, and an alternate site storage cell (not a virtual disk) for the destination.
  • the destination virtual disk does not exist prior to the copy set creation operation. Instead, it is created specifically to be a member of the copy set. Because each destination or alternative virtual disk is created on demand, it is created specifically and automatically to be compatible with the copy set. Hence, there is little or no manual procedures that must be performed to create new members for a copy set. This allows the copy set to increase (or decrease) its membership readily.
  • a LUN is identified to a host 102 by a WWLID.
  • a WWLID is identified to a host 102 using the same WWLID.
  • the LUN appears to the host 102 as a single LUN that can be accessed by multiple paths.
  • each controller 105 that manages one of the LUNs in a copy set can be queried by a host 102 to determine a particular LUN or controller 105 (identified by the UUID) that is preferred (but not mandatory) for use by that host for access to the LUN.
  • This feature supports large fan out in that any number of LUNs in a copy set can be presented as a single LUN with replication between the LUNs handled automatically by controllers 105 without involving hosts 102 .
  • a controller 105 at the destination site determines whether its storage cell has sufficient resources (e.g., storage capacity) to service the request, reserves the necessary resources if available, and responds back to the controller 105 at the source site that the destination virtual disk is created.
  • An existing virtual disk cannot be used for the destination. This process is very quick as the destination virtual disk is not actually allocated, and the data is not actually copied from source to destination at this time.
  • the source virtual disk 601 receives this confirmation, it can continue normal operations while the actual allocation of virtual disk 602 and copying of data can occur as background processes. Hence, the actual time that source virtual disk 602 is quiesced during the data replication initialization is a few milliseconds.
  • a controller may support tens, hundreds, or thousands of copy sets, limited only by the practical constraint of the computing resources (e.g., memory and processing power) needed to manage the number of copy sets.
  • a path to the alternate site is found and a connection is established.
  • This connection need not be a persistent connection, although for data that frequently changes, a persistent connection may be efficient.
  • heartbeat is initiated over the connection.
  • Heartbeats are in both directions, meaning the source site generates a heartbeat on each connection, and the alternate site(s) generates a heartbeat on each connection.
  • Heartbeat timeout intervals are adaptive based on distance (computed round trip delay).
  • a controller that is used to create the source or destination virtual disks is initially preferred, which constrains the number of available paths initially.
  • the chosen path/connection is used until it breaks, then an attempt is made to find another path.
  • the port/path selection algorithm may include other default designations such as a preferred fabric or switch, or preferred ports on controllers, or similar default preferences that may operate to select among redundant resources.
  • objects do not span sites or storage controllers. Instead, objects exist at each site and are associated with each other as suggested in FIG. 7 .
  • the controller views a “DRM Group” 705 as a source group associated with a destination group (represented by a dashed-line oval in FIG. 7 ).
  • the controller does not have a copy set object per se, rather group members (virtual disks) that are associated with a remote member's (virtual disks).
  • the copy set is represented by this association.
  • virtual disks 701 are associated with virtual disks 702 to form a copy set
  • virtual disks 711 are associated with virtual disks 712 to form another copy set.
  • DRM groups 705 comprise a set of related virtual disks or LUNs that belong to copy sets all of which have the same source and destination. Multiple groups 705 can exist for a given source and destination combination. While in one sense a group 705 spans sites, in practice a separate logical object that represents a group is implemented in each site (i.e., each controller 105 managing a group). Hence, in another sense each group 705 comprises a set of source halves of copy sets (i.e., the source LUNs 701 and 711 ), or destination halves of copy sets (i.e., destination LUNs 702 and 712 ).
  • a group 705 is considered a “local group” by the controller 105 that implements the group logical object, and is considered a “remote group” when implemented by another controller 105 . In other words, whether a group 705 is local or remote is determined from the perspective of a particular controller 105 .
  • a local group 105 is associated with a remote group 105 when the LUNs contained in each group are in the same copy set. In a multidirectional implementation, each local group 705 will be associated with a plurality of remote groups 105 .
  • a group contains a collection of LUNS for which a combined I/O stream must be replicated in the LUNs of each remote group to provide certain properties with regard to its interruption (e.g. a copy set holding a database and a copy set holding a journal related to the database).
  • DRM groups 705 are used for maintaining crash consistency and preserving WRITE ordering.
  • Crash consistency refers operation in the event that a host 102 or link to a host 102 crashes during a data transaction. In such event, it is important that the data transaction be committed to all members of the group or fail in all members of the group.
  • a database/journal implementation for example, it is important to prevent a situation in which the database is modified, but the journal does not reflect the modification, or vice versa.
  • the consistency property applies when the group has more than one member. In general, more than one virtual disk should be added to a group only if the application requires it. In most cases, a group should comprise only one member for optimal performance, in which cases the crash consistency behaviors are less important
  • a group maintains write ordering among the members for asynchronous operation and logging/merging.
  • Asynchronous operation refers to an operation mode in which a modification to one member of a copy set can be propagated to other members of the copy set after a time delay. During this time delay, the various replicas are inexact.
  • asynchronous operation it is important that all replicas eventually implement the modification. Since multiple modification operations may be pending but uncommitted against a particular replica, it is necessary that the original order in which the modifications were presented be preserved when the pending modifications are applied to each replica. Even where asynchronous operation is not explicitly allowed, a destination LUN may become unavailable for a variety of reasons, in which case a copy set is implicitly operating in an asynchronous mode.
  • a record is maintained in a non-volatile storage device such as a cache for each group 705 that records the history of write commands and data from a host.
  • the record is sized to store all write transactions until the transaction is committed to each member of a copy set.
  • the record can be written to a log on media-based storage and then later be replayed to merge the pending writes, in order, to each remote group 705 .
  • the ordering algorithm uses a “group sequence number” and the remote groups 705 ensure that the data is written in order sequence. Group members enter and exit logging at the same time, to assure order across the volumes.
  • Virtual disks in a group should have the same alternate site.
  • a group is limited to 32 virtual disks in a particular implementation.
  • a virtual disk can belong to at most one group.
  • Virtual disks in the same DRM group 705 may belong to different disk groups.
  • the controller automatically creates a symmetric group object on the alternate site controller.
  • a group is created during copy set creation, if the user chooses not to use an existing group.
  • all members of a group are accessed on the same controller in a dual pair to enforce cache coherency (i.e., not split between dual controllers).
  • the preferred storage controller should be the same on all members. When members are added to a group, they are automatically moved to reside on the same controller, and thereafter will failover together. If the preferred storage controller is changed on one member, it will change the setting for all members.
  • One benefit of the flexible architecture described above is that the number of replicas in a copy set can be increased. As noted hereinbefore, conventional systems allow only one replica in a given copy set. In accordance with the present invention, any number of replicas can be included in a copy set. This is because roles can be switched dynamically between source and destination which eases propagation of data across multiple members. Also, because the storage at each site is virtualized, LUNs of any desired size and configuration can be automatically configured from physical storage capacity available at that site. Accordingly, there is no need to specially configure hardware or software at each site in order to implement a LUN that is compatible with a given copy set.
  • each controller 105 can automatically increase the size or change the configuration of the LUN(s) it manages, the manual tasks involved to increase the size of a LUN do not grow geometrically or exponentially with the number of replicas in the copy set. Instead, the user simply manages the LUN as a single entity, and the implementation of changes in each replica is handled automatically.
  • system of the present invention provides efficiency in that redundant components can operate simultaneously and in parallel to provide improved throughput as well as serving during failover to replace functionality of failed systems and connections. Because system components have few constraints on their functionality, the present invention allows great flexibility in implementing a particular system to emphasize any balance of, for example, low latency, high availability, disaster tolerance, and cost.

Abstract

A data replication management (DRM) architecture comprising a plurality of storage cells interconnected by a fabric. Flexibility in connectivity is provided by configuring each storage cell port to the fabric to handle both host data access requests and DRM traffic. Each storage cell comprises one or more storage controllers that can be connected to the fabric in any combination. Processes executing in the storage controller find a path to a desired destination storage cell. The discovery algorithm implements a link service that exchanges information related to DRM between the storage controllers. The DRM architecture is symmetric and peer cooperative such that each controller and storage cell can function as a source and a destination of replicated data. The DRM architecture supports parallel and serial “fan-out” to multiple destinations, whereby the multiple storage cells may implement data replicas.

Description

    FIELD OF THE INVENTION
  • The present invention relates generally to error recovery in data storage systems, and more specifically, to a system for providing controller-based remote data replication using a redundantly configured Fibre Channel Storage Area Network to support data recovery after an error event and enhance data distribution and migration.
  • BACKGROUND OF THE INVENTION AND PROBLEM
  • Recent years have seen a proliferation of computers and storage subsystems. Demand for storage capacity grows by over seventy-five percent each year. Early computer systems relied heavily on direct-attached storage (DAS) consisting of one or more disk drives coupled to a system bus. More recently, network-attached storage (NAS) and storage area network (SAN) technology are used to provide storage with greater capacity, higher reliability, and higher availability. The present invention is directed primarily SAN systems that are designed to provide shared data storage that is beyond the ability of a single host computer to efficiently manage.
  • Mass data storage systems are implemented in networks or fabrics that provide means for communicating data between systems that use data, and the storage systems that implement the physical storage. In many cases, host computers act as storage servers and are coupled to the network and configured with several disk drives that cumulatively provide more storage capacity or different storage functions (e.g., data protection) than could be implemented by a DAS system. For example, a server dedicated to data storage can provide various degrees of redundancy and mirroring to improve access performance, availability and reliability of stored data. Collecting storage sub-systems, where a separate server manages each sub-system, can form a large storage system. More recently, virtualized storage systems such as the StorageWorks® Enterprise Virtual Array announced by Compaq Corporation in October, 2001 provide storage controllers within a fabric or network that present virtualized storage to hosts that require data storage in a manner that enables the host to be uninvolved in the physical configuration, allocation and management of the storage devices. StorageWorks is a registered trademark of Compaq Computer Corporation. In this system, hosts simply access logical units of storage that appear to the host as a range of logical address space. Virtualization improves performance and utilization of storage.
  • SAN systems enable the possibility of storing multiple copies or “replicas” of data at various physical locations throughout the system. Data replication across multiple sites is desirable for a variety of reasons. To provide disaster tolerance, copies of data stored at different physical locations is desired. When one copy becomes unavailable due to equipment failure, a local network outage, natural disaster or the like, a replica located at an alternate site can allow access to the data. Replicated data can also theoretically improve access in normal operation in that replicas can be accessed in parallel, avoiding bottlenecks associated with accessing a single copy of data from multiple systems. However, prior systems were organized such that one site had a primary role and another site was a replica. Access requests were handled by the primary site until failure, at which time the replica became active. In such architecture, the replica provided little benefit until failure. Similarly, the resources allocated to creating and managing replicas provided minimal load balancing benefit that would enable data access requests to be directed intelligently to replicas such that resources were used more efficiently. Moreover, when multiple replicas are distributed throughout a network topology, it would be beneficial if network delays associated with accessing a topologically remote storage subsystem could be lessened.
  • In the past, managing a data replication system required significant time and expense. This time and expense was often related to tasks involved in setting up and configuring data replication on a SAN. Physical storage devices between original and replica locations had to be closely matched which could require knowledge at the spindle level to set up a storage site to hold a replica. Similarly detailed knowledge of the physical devices at a storage site were required to set up logging of replication operations. Moreover, the logical structures used to represent, access and manage the stored data had to be substantially identically reproduced at each storage site. Many of these operations required significant manual intervention, as prior data replication architectures were difficult to automate. This complexity made it difficult if not impossible to expand the size of a replicated volume of storage, as the changes on one site needed to be precisely replicated to the other site. A need exists to provide data replication systems in a SAN that enable functions involved in setup and configuration of a replication system to be automated, and allow the configuration to be readily expanded.
  • It is desirable to provide the ability for rapid recovery of user data from a disaster or significant error event at a data processing facility. This type of capability is often termed ‘disaster tolerance’. In a data storage environment, disaster tolerance requirements include providing for replicated data and redundant storage to support recovery after the event. In order to provide a safe physical distance between the original data and the data to be backed up, the data is migrated from one storage subsystem or physical site to another subsystem or site. It is also desirable for user applications to continue to run while data replication proceeds in the background. Data warehousing, ‘continuous computing’, and enterprise applications all benefit from remote copy capabilities.
  • Compaq Corporation introduced a data replication management product in its Array Controller Software (ACS) operating on an HSG80 storage controller and described in U.S. patent application Ser. No. 09/539,745 assigned to the assignee of the present application and incorporated herein by reference. This system implemented architecture with redundant storage controllers at each site. Two sites could be paired to enable data replication. While effective, the HSG80 architecture defined relatively constrained roles for the components, which resulted in inflexibility.
  • For example, each of the controllers comprised one port that was dedicated to user data, and a separate port that was dedicated to data replication functions. Even where redundant fabrics were implemented, for a given controller both of these ports were coupled to a common fabric switch. Despite the fact that each controller had two ports for communicating with other controllers, one of the ports was constrained in the role of handling user data, and the other port was constrained in the role of handling data replication. Failure of either port would be, in effect, a failure of the entire controller and force migration of storage managed by the failed controller to the redundant controller. Similarly, failure of a communication link or fabric coupled to one port or the other would render the controller unable to perform its tasks and force migration to the redundant controller. Such migration was disruptive and typically required manual intervention and time in which data was unavailable.
  • As another example, prior data replication management solutions simplified the implementation issues by assigning fixed roles to storage locations. A particular storage site would be designated as a primary when it handled operational data traffic, and another site would be designated as a secondary or backup site. Such architectures were unidirectional in that the backup site was not available for operational data transactions until the failure of the primary site. Such rigidly assigned roles limited the ability to share storage resources across multiple topologically distributed hosts. Moreover, configuration of such systems was complex as it was necessary to access and program storage controllers at both the primary and secondary sites specifically for their designated roles. This complexity made it impractical to expand data replication to more than two sites.
  • This lack of flexible configuration results in constraints imposed on the configuration and functionality of DRM implementations. Most existing data replication solutions have specific constraints around the number of places that the data may be copied, the simultaneity of multi-directional copies. Further, the specific nature of the synchronicity of the data transmission between sites was per controller, not per volume. As a result, all of the copy sets managed by a particular controller had the exact same initiator and target role designations. Also, all copy sets had to go in one direction such that if a controller was an initiator for one copy set, it was an initiator for all copy sets managed by that controller. Further, the replicas were not allowed to vary from the original in any material respect. The target disks were required to have the same size, data protection scheme, and the like as the original. Prior systems could not readily support dynamic changes in the size of storage volumes.
  • The lack of flexible configuration constrains the number of replicas that can be effectively created. Current systems allow an original data set to be replicated at a single location. While a single replica is beneficial for disaster tolerance, it is of limited benefit to improving performance benefits from migrating or distributing data to locations closer to where the data is used. A need exists for a data replication system that improves the ability to fan-out a larger number of replicas to improve geographic and topological diversity.
  • Therefore, there remains a need in the art for a data storage system capable of providing flexible data replication services without the direct involvement of the host computer. Moreover, a data storage system is needed that is readily extensible to provide multiple replication, load balancing, and disaster tolerance without limitations imposed by designating rigid roles for the system components.
  • SUMMARY OF THE INVENTION
  • Briefly stated, the present invention involves architecture for flexible data replication management comprising first and second storage cells at separate topological locations on a storage area network. Each storage location comprises a storage controller having first and second ports coupled to a communication network, and a pool of storage devices accessible through the controller. Each of the first and second ports communicates both user data and replication data with the communication network. Preferably, each storage location is enabled to operate as a source that initiates replication of data stored within its pool of storage devices, and as a destination for storing replicas of data stored within storage devices of another storage location. Preferably, the communication network comprises redundant sub-networks, and each controller is coupled such that the first port couples to one of the sub-networks and the second port couples to another of the sub-networks.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • FIG. 1 shows a storage area network (SAN) environment in which the present invention is implemented;
  • FIG. 2 illustrates an alternative SAN environment embodying the present invention;
  • FIG. 3 shows an implementation emphasizing redundant connections to improve disaster tolerance;
  • FIG. 4 illustrates an implementation with less redundancy than the implementation of FIG. 3;
  • FIG. 5 illustrates a basic implementation of the present invention;
  • FIG. 6 shows data flow relationships in a data replication management operation in accordance with the present invention; and
  • FIG. 7 illustrates logical relationships between data structures of the present invention.
  • DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
  • In general, the present invention describes a data replication management (DRM) architecture comprising a plurality of storage cells interconnected by a data communication network such as a fibre channel fabric. The present invention emphasizes symmetry and peer-cooperative relationships amongst the elements to provide greater flexibility in configuration and operation. Storage cells are for the most part autonomous units of virtualized storage that implement hundreds of gigabytes to terabytes of storage capacity. In accordance with the present invention, each storage cell can act as a source or primary location for data storage, and at the same time act as a destination or secondary location holding a replica data from a primary location of another storage cell. Similarly, the architecture of the present invention seeks to minimize rigid roles placed on interface ports and connections such that all available connections between components can support both host data traffic (e.g., traffic involving data that is to be stored and retrieved), and DRM traffic (e.g., traffic related to managing replicas between storage cells). In this manner, resources are efficiently deployed and operational performance is improved.
  • The present invention is described with various levels of specificity to ease description and understanding. However, unless specified otherwise, the specific implementations are examples only, and not limitations of the invention. For example, network connectivity is illustrated by fibre channel mechanisms, however, other network mechanisms provide suitable functionality in particular environments. In particular implementations, storage capacity is presented as SCSI (small computer system interface) logical units, although the particular protocol is readily varied to meet the needs of a particular application.
  • FIG. 1 illustrates a multi-site storage area network (SAN) implementing features in accordance with present invention. The example shown in FIG. 1 includes only three sites (Site 1, Site 2 and Site 3) for to ease illustration and understanding, however, it should be understood that any number of sites may be provided to meet the needs of a particular application. The various sites may be physically proximate, or separated by an arbitrary distance. A particular implementation may well limit the number of possible sites, and may well limit the maximum or minimum physical separation of sites. Each site includes one or more storage cells 101, such as cells 101A, 101B, 101C and 101D. Any number of storage cells 101 maybe included in any site, although the number implemented in any particular application may be constrained to meet the needs of that application.
  • The storage implemented at various sites is accessed by host computer 102, such as host computers 102A, 102B and 102C. Host computers are generally machines that consume or require data storage. Typical host computers 102 demand large quantities of storage such as mainframes, web servers, transaction processors, and the like. However, a host computer 102 may comprise a computer of any processing capacity that requires or benefits from network storage either to reduce cost of the host 102, implement more storage capacity than practical in a host 102, share data amongst multiple hosts 102, or the like. A host computer 102 may couple to the storage cells 101 via a connection to network 103 such as illustrated by host computers 102A and 102B. In many cases, a host computer 102 will be located in a site such as host computer 102C located in site 2. The topological location of host computers 102 is a matter of design choice selected to meet the needs of the particular application. In many cases, one or more host computers 102 will be located at a site. It should be appreciated, however, that host computers 102 have little to do with the management and configuration of the DRM system of the present invention, other than the reality that as consumers of data, the DRM system is preferably able to satisfy storage needs of host computers 102 at various physical and topological locations.
  • Storage cells 101 and hosts 102 couple to data communication network 103. Storage cells 101 implement a quantity of data storage capacity that is accessible through storage controllers 105 that implement one or more connections to network 101. Storage cells 101 typically implement hundreds of gigabytes to terabytes of physical storage capacity. Preferably, controllers 105 virtualize the physical storage capacity such that it is configurable into logical units (LUNs) of storage capacity. The LUNs implement an arbitrary quantity of logical address block storage, where each LUN has a specified level of data protection such as RAID 0-5 data protection.
  • The logical construction or context of a storage cell 101 comprises the logical and data structures that are used to represent, manage and configure the raw physical storage capacity into devices that can be accessed by hosts 102 in a desired manner. To do this, controllers 105 implement and manage various logical objects that implement data structures and behavior to represent various components of each storage cell 101, preferably without involvement if hosts 102. All objects in the system are identified with a universally unique identification (UUID). The UUIDs are stored persistently in metadata structures within the physical storage which enables all or part of a storage cell, and all the LUNs implemented therein, to be mapped and presented by any storage controller 105. In a data replication application, the ability to present the LUNs and the context in which they exist is useful, as described in greater detail below, in event of a controller failure at one of the replica sites.
  • Of particular significance to the present invention is a logical object representing each LUN that is presented to a host system 102. In addition to the UUID, LUNs are also identified by a worldwide LUN ID (WWLID) which identifies the LUN to hosts 102. Hosts 102 access physical storage capacity by addressing read and write operations to specified LUNs using the WWLID, and can be otherwise unaware of the physical storage architecture or data protection strategy for a particular LUN that is being accessed. Storage controllers 105 manage the tasks of allocating physical storage capacity to specified LUNs, monitoring and maintaining integrity of the LUNs, moving data between physical storage devices, and other functions that maintain integrity and availability of the data stored therein.
  • Network 103 comprises any of a variety of available networks, and may comprises a plurality of interconnected networks. In particular examples, network 103 comprises at least two independent fibre channel fabrics to provide redundancy. These fibre channel fabrics may comprise long-distance connection mechanism 201 such as asynchronous transfer mode (ATM) and Internet protocol (IP) connections that enable sites to be separated by arbitrary distances.
  • At least one SAN management appliance (SMA) 107 is coupled to network 103 to enable connections to storage cells 101. In practice, a number of SMAs 107 are provided, and typically an SMA 107 is located at each site to provide management operation for the storage cells 101 at that site. However, because each SMA 107 communicates through network 103, the physical location of SMAs 107 is arbitrary. SMAs 107 are preferably implemented at topological locations that provide redundant connections to the storage cells 101.
  • Network 103 is accessible directly or indirectly to all components at Site 1, Site 2, and Site 3 including hosts 102 and controllers 105. In preferred implementations, each component has redundant links to network 103, and network 103 is preferably implemented as having redundant sub-networks as shown in FIG. 2. Redundancy provides connectivity in event of failure or degradation of some portions of network 103. Redundancy also enables connectivity in event of failure or degradation of controllers 105 and/or interface components of hosts 102. In prior data replication systems, it was known to provide multiple channels between components, however, these channels were not interchangeable. For example, each controller might have two host ports to a network, however, one of the ports was designed to support operational data traffic while the other port was dedicated to data replication operations. While this division of responsibility was simpler to implement, it increased the likelihood that one or the other of the ports would be used at less than full capacity. Moreover, special purpose ports are not completely redundant, hence the system would be vulnerable to failure of one or the other of the channels. Even where two controllers were provided, this asymmetry between ports affected performance.
  • In the systems of FIG. 1 and FIG. 2, it is important to realize that storage cell 101 is able to function as a primary storage location for any of hosts 102, and at the same time function as a secondary or alternative storage location for a replica of data from another storage cell 101. This feature is referred to as “bi-directionality”. Unlike prior DRM systems where sites were rigidly configured as either primary or secondary sites, the present invention provides for flexible assignment at a much finer level of granularity, preferably such that individual virtual disks can be designated as either primary or alternate. One effect of such flexibility is that it becomes practical to implement enormous quantities of storage in each storage cell 101 as the capacity at each site can be readily applied to various applications with disparate storage needs.
  • FIG. 3 illustrates various features of the present invention embodied in a disaster tolerant configuration. The configuration of FIG. 3 emphasizes redundancy in physical components. As a result, multiple paths exist between any host and any storage. Storage management appliances 107 include a Bridge and a data replication management console (DRMC) which are cooperative software processes that each present a user interface for configuration and monitoring of DRM processes. The Bridge functions as an element manager for controllers 105, and handles basic functions of connecting to the communication network 103, issuing commands to controllers 105, and implementing security policies. All customer configuration and monitoring of DRM activities goes through the Bridge. The DRMC is provides increased functionality over the Bridge interface, by managing solely DRM storage and providing assists, such as with site recovery.
  • One goal of a disaster tolerant configuration in FIG. 3 is to provide hardware redundancy for multiple levels of fault tolerance in order to keep the primary and alternate sites running, almost at ‘non-stop’ metrics (seconds per year downtime, barring disaster or planned downtime). In the face of single component failures at a site, DRM will failover to a redundant component at that site to allow continued operation. If a significant failure happens to the primary site, then data processing can be resumed at the alternate site where the data is intact. Database or application recovery is achieved with little or no downtime.
  • In this regard, wherever redundant components are provided, they are preferably interchangeable in accordance with the present invention such that functionality of one component can be taken over by a redundant component. For example, node ports in the SMA 107 are redundant such that failure of one node port can be handled by moving functions handled by that port over to the other, non-failed node-port.
  • Physical storage devices 302 are coupled to controllers 105 by a high-speed redundant connection such as fibre channel arbitrated loop (FCAL) connections 303 in FIG. 3. An FCAL connection allows each of tens or hundreds of individual hard disk drives to be accessed not only at high speed, but by either controller 105 in a storage cell. Further, by providing redundant FCALs, one FCAL can fail while each storage device 302 remains accessible by either controller. The present invention is largely independent of the specific implementation of physical storage devices 302, and it is contemplated that other physical storage architectures will provide suitable equivalents for purposes of the present invention.
  • FIG. 3's disaster tolerance configuration provides for high availability with a dual fabric implemented by switches 301, dual hosts 102, and dual storage topology, where a single switch, host, or storage can fail and the system can still continue access on the SAN. Each fabric is composed of two switches 301, with the switches connected to each other over what is called an E-port, or expansion port. The E-port can be a connection with distances that vary depending on technology. Fibre Channel allows for 10km and with extenders up to 100 km. ATM provides for ‘round the world’ distances. New technology, called FC-BB (Fibre Channel Backbone), provides the opportunity to extend Fibre Channel over leased Telco lines (also called WAN tunneling). There is no theoretical constraint imposed on the number of switches 301, although a particular implementation may choose to impose such a constraint. With more switches 301, the fabric 103 becomes more complex and port-to-port communications may require more “hops”. Hence, both the latency and the variability in latency increase with more complex fabrics.
  • Hosts 102 preferably run multi-pathing software that dynamically allows failover between storage paths as well as static load balancing of storage volumes (LUNs) between the paths of controllers 102. Multi-pathing software enables a host 102 to identify multiple connection options that lead to desired storage, and select amongst the available paths based on selected criteria. These criteria may vary widely, but typically include path availability, path performance, path load, controller load, and the like. This allows for applications to continue given any failure of fabric or the controllers. A lower cost variant of this configuration uses a fabric comprising only one switch, such that there is no E-port. Such a configuration may be appropriate for shorter distances.
  • FIG. 4 illustrates a configuration of the present invention that emphasizes data movement rather than redundancy of the example in FIG. 3. The configuration of FIG. 4 provides a lower cost solution ideal for customers only interested in data backup, data migration, data distribution, and data mining. FIG. 4 has a single fabric formed by two interconnected switches. With shorter distances, an even lower cost variant could use only one switch, such that there is no E-port. FIG. 5 illustrates an even more basic configuration using a fabric having a single switch 301, one host 102, and non-redundant controllers 105.
  • Because ports of controllers 105 are configured to handle both operational data transactions and DRM traffic, a single port connection between each controller 105 and switch 301 is all that is necessary to implement data replication features of the present invention. However, the basic configuration shown in FIG. 5 sacrifices much redundancy, and would not be recommended for most applications. It is important to understand that a great deal of flexibility exists in configuring the present invention for specific applications. Redundant connections and components can be added and removed flexibly to meet the needs of a particular application. Hence, various features of the configurations shown in FIG. 1 through FIG. 5 can be mixed and matched to configure a specific implementation.
  • A SAN, such as shown in FIG. 1 through FIG. 5, is established by coupling controllers 105 and hosts 102 to switch 301, or a fabric formed from multiple switches 301. Each of the devices performs a log in operation according to applicable fibre channel standards. The log in operation involves an exchange of information in which each device is registered with the fabric 103, and devices can discover other devices that are registered with fabric 103. In the preferred implementations, each controller 105 is identified by a storage cell unique identification number (UUID) which is stored by a name server in fabric 103. By way of this registration and discovery process, each controller 105 can obtain knowledge necessary to connect to any other controller 105.
  • As noted before, metadata needed to reconstruct a storage cell 101 including the UUIDs of the various objects in that storage cell is stored persistently. Because of this, in the event of a destination controller failure, another controller 105, typically a redundant or partner controller 105 in the same storage cell 101 as the failed controller 105, can reconstruct the storage cell 101. Other controllers 10 involved in a copy set can continue to locate the now reconstructed storage cell 101, and continue data replication operations without loss of data or requiring manual processes. The ability to implement this level of failure recovery substantially or wholly automatically is unprecedented.
  • In the event of a connection failure between a source controller 105 and a destination controller 105, the source controller 105 can readily identify each available alternate controller 105 and continue operation without loss of data or requiring manual intervention. A connection failure or link failure is distinct from a controller failure in that the controllers 105 remain operational, but unable to maintain a data communication link with one or more other controllers 105. In response to a connection failure, the source controller has several options, depending on the nature of the failure. When the source controller 105 is unable to link to fabric 103, suggesting a failure in a port of switch 301, the controller 105 can attempt to use an alternate link to fabric 103, or a redundant fabric 103 if available. When the link between fabric 103 and a destination controller 105 has failed, the source controller 105 attempts to establish a new link with a redundant port of the destination controller 105. In both of these cases, there will be no need to reconstruct the storage cell 101, and data replication operations will proceed with substantially no interruption, and in most cases without any need to notify the host 102 of the aberrant condition. In contrast, prior systems had only one port on each controller available for DRM operations and so a link failure would require reconstruction of either the source or destination LUN in an alternate controller.
  • In some cases a link failure may require migration of a LUN from one controller 105 to another. For example, if all ports on either the destination or source 5 controllers 105 failed (which would typically be considered a controller failure), or several ports to fabric 103 failed simultaneously in particular combination, recovery would require migrating presentation of one or more LUNs from one controller 105 to a redundant or partner controller 105 in a manner similar to a controller failure event. This migration can be performed efficiently using the mechanisms for mapping and presenting a storage cell 101 described above. Notably, all LUNs do not need to be presented before pending data replication operations can be completed. In implementations where the LUNs are virtualized, as described in co-pending patent application Ser. No. 10/040,194 entitled “SYSTEM AND METHOD FOR ATOMIZING STORAGE” filed on Oct. 22, 2001 and which is assigned to the assignee of the present invention and incorporated herein by reference, it is possible to implement only so much of the data structures needed to represent the portion of the LUN that is the subject of a pending data transaction.
  • FIG. 6 shows hierarchical relationships defined in accordance with the present invention to model data replication management. Prior to performing DRM operations, a storage cell 101 is implemented at each site, and a virtual disk 601 is allocated within a storage cell 101 that is handing operational data with one or more hosts 102. For any particular copy set, one virtual disk 601 is designated as a source, although it should be understood that a source virtual disk 601 might be allocated at any site implementing the DRM features of the present invention. Virtual disk 601 may comprise a plurality of physical storage resources that span multiple physical drives within the storage cell 101, and may implement any desired capacity and data protection type.
  • A destination virtual disk 602 is allocated within a storage cell 101 at a designated alternative or destination site. In normal operation, write operations are directed to source virtual disk 601, and copied in a background process to one or more destination virtual disks 602. Destination virtual disk has the same logical storage capacity as the source virtual disk, but may provide a different data protection configuration. Controllers 105 of the destination storage cell handle the allocation of destination virtual disk 602 autonomously. This allocation involves creating data structures that map logical addresses to physical storage capacity, and in a particular implementation involve processed described in greater detail in U.S. patent application Ser. No. 10/040,194.
  • The actual allocation and copying of data may take up to several minutes to several hours in the case of storage of many gigabytes or terabytes. These processes can be streamlined by allocating and copying only logical storage blocks that are used in source virtual disk 601. For example, a virtual disk 601 may implement 1 terabyte of storage, but a sparsely populated example may use only a few gigabytes of capacity to store actual data. In accordance with the present invention, destination virtual disk 602 will reserve resources needed to implement the entire 1 terabyte, but will only allocate and copy the locations that are actually used to store data and may allocate the other locations. This greatly reduces the time required to create replicates.
  • A “copy set” is a set of member virtual disks where each member virtual disk is a replica of the others, and may comprise any number of replica virtual disks. While it may be easier to describe and understand the present invention by designating one virtual disk as a source or original virtual disk and the others as destinations or replicas, it is important to understand that once the copy set is created each member virtual disk is essentially a peer of all others. To create a copy set, the user designates a virtual disk as the source, and an alternate site storage cell (not a virtual disk) for the destination. The destination virtual disk does not exist prior to the copy set creation operation. Instead, it is created specifically to be a member of the copy set. Because each destination or alternative virtual disk is created on demand, it is created specifically and automatically to be compatible with the copy set. Hence, there is little or no manual procedures that must be performed to create new members for a copy set. This allows the copy set to increase (or decrease) its membership readily.
  • As noted before, a LUN is identified to a host 102 by a WWLID. One feature of a particular embodiment of the present invention is that each LUN in a copy set is presented to hosts 102 using the same WWLID. In this manner, the LUN appears to the host 102 as a single LUN that can be accessed by multiple paths. Preferably, each controller 105 that manages one of the LUNs in a copy set can be queried by a host 102 to determine a particular LUN or controller 105 (identified by the UUID) that is preferred (but not mandatory) for use by that host for access to the LUN. This feature supports large fan out in that any number of LUNs in a copy set can be presented as a single LUN with replication between the LUNs handled automatically by controllers 105 without involving hosts 102.
  • Initially, when a controller 105 at the destination site is requested to create a destination virtual disk, it determines whether its storage cell has sufficient resources (e.g., storage capacity) to service the request, reserves the necessary resources if available, and responds back to the controller 105 at the source site that the destination virtual disk is created. An existing virtual disk cannot be used for the destination. This process is very quick as the destination virtual disk is not actually allocated, and the data is not actually copied from source to destination at this time. Once the source virtual disk 601 receives this confirmation, it can continue normal operations while the actual allocation of virtual disk 602 and copying of data can occur as background processes. Hence, the actual time that source virtual disk 602 is quiesced during the data replication initialization is a few milliseconds. It is also possible that accesses to source virtual disk 601 can be cached or buffered during the initialization process so that no downtime is observed from the hosts 102. A controller may support tens, hundreds, or thousands of copy sets, limited only by the practical constraint of the computing resources (e.g., memory and processing power) needed to manage the number of copy sets.
  • To implement a copy transaction between a source and destination, a path to the alternate site is found and a connection is established. This connection need not be a persistent connection, although for data that frequently changes, a persistent connection may be efficient. Preferably, heartbeat is initiated over the connection. Heartbeats are in both directions, meaning the source site generates a heartbeat on each connection, and the alternate site(s) generates a heartbeat on each connection. Heartbeat timeout intervals are adaptive based on distance (computed round trip delay).
  • An attempt is made to choose paths by not moving the units between controllers. In other words, a controller that is used to create the source or destination virtual disks is initially preferred, which constrains the number of available paths initially. The chosen path/connection is used until it breaks, then an attempt is made to find another path. The port/path selection algorithm may include other default designations such as a preferred fabric or switch, or preferred ports on controllers, or similar default preferences that may operate to select among redundant resources.
  • In the preferred implementation, objects do not span sites or storage controllers. Instead, objects exist at each site and are associated with each other as suggested in FIG. 7. For instance, the controller views a “DRM Group” 705 as a source group associated with a destination group (represented by a dashed-line oval in FIG. 7). The controller does not have a copy set object per se, rather group members (virtual disks) that are associated with a remote member's (virtual disks). The copy set is represented by this association. In FIG. 7, virtual disks 701 are associated with virtual disks 702 to form a copy set and virtual disks 711 are associated with virtual disks 712 to form another copy set.
  • DRM groups 705 comprise a set of related virtual disks or LUNs that belong to copy sets all of which have the same source and destination. Multiple groups 705 can exist for a given source and destination combination. While in one sense a group 705 spans sites, in practice a separate logical object that represents a group is implemented in each site (i.e., each controller 105 managing a group). Hence, in another sense each group 705 comprises a set of source halves of copy sets (i.e., the source LUNs 701 and 711), or destination halves of copy sets (i.e., destination LUNs 702 and 712). A group 705 is considered a “local group” by the controller 105 that implements the group logical object, and is considered a “remote group” when implemented by another controller 105. In other words, whether a group 705 is local or remote is determined from the perspective of a particular controller 105. A local group 105 is associated with a remote group 105 when the LUNs contained in each group are in the same copy set. In a multidirectional implementation, each local group 705 will be associated with a plurality of remote groups 105.
  • In essence, a group contains a collection of LUNS for which a combined I/O stream must be replicated in the LUNs of each remote group to provide certain properties with regard to its interruption (e.g. a copy set holding a database and a copy set holding a journal related to the database). DRM groups 705 are used for maintaining crash consistency and preserving WRITE ordering. Crash consistency refers operation in the event that a host 102 or link to a host 102 crashes during a data transaction. In such event, it is important that the data transaction be committed to all members of the group or fail in all members of the group. In a database/journal implementation, for example, it is important to prevent a situation in which the database is modified, but the journal does not reflect the modification, or vice versa. The consistency property applies when the group has more than one member. In general, more than one virtual disk should be added to a group only if the application requires it. In most cases, a group should comprise only one member for optimal performance, in which cases the crash consistency behaviors are less important.
  • A group maintains write ordering among the members for asynchronous operation and logging/merging. Asynchronous operation refers to an operation mode in which a modification to one member of a copy set can be propagated to other members of the copy set after a time delay. During this time delay, the various replicas are inexact. When asynchronous operation is allowed, it is important that all replicas eventually implement the modification. Since multiple modification operations may be pending but uncommitted against a particular replica, it is necessary that the original order in which the modifications were presented be preserved when the pending modifications are applied to each replica. Even where asynchronous operation is not explicitly allowed, a destination LUN may become unavailable for a variety of reasons, in which case a copy set is implicitly operating in an asynchronous mode.
  • To ensure write order preservation, a record is maintained in a non-volatile storage device such as a cache for each group 705 that records the history of write commands and data from a host. The record is sized to store all write transactions until the transaction is committed to each member of a copy set. When required, the record can be written to a log on media-based storage and then later be replayed to merge the pending writes, in order, to each remote group 705. The ordering algorithm uses a “group sequence number” and the remote groups 705 ensure that the data is written in order sequence. Group members enter and exit logging at the same time, to assure order across the volumes.
  • Virtual disks in a group should have the same alternate site. A group is limited to 32 virtual disks in a particular implementation. A virtual disk can belong to at most one group. Virtual disks in the same DRM group 705 may belong to different disk groups. When a group object is created on the primary site controller, the controller automatically creates a symmetric group object on the alternate site controller. A group is created during copy set creation, if the user chooses not to use an existing group.
  • At a site, all members of a group are accessed on the same controller in a dual pair to enforce cache coherency (i.e., not split between dual controllers). The preferred storage controller should be the same on all members. When members are added to a group, they are automatically moved to reside on the same controller, and thereafter will failover together. If the preferred storage controller is changed on one member, it will change the setting for all members. These constraints create a configuration in which all members in a group share the same connection/path between source and destination sites. Different groups may share the same connection/path to the same alternate site or a different one. If more than one member in a group needs to copy, then they are copied one at a time with a queue depth selected to keep the pipe full for optimal performance.
  • One benefit of the flexible architecture described above is that the number of replicas in a copy set can be increased. As noted hereinbefore, conventional systems allow only one replica in a given copy set. In accordance with the present invention, any number of replicas can be included in a copy set. This is because roles can be switched dynamically between source and destination which eases propagation of data across multiple members. Also, because the storage at each site is virtualized, LUNs of any desired size and configuration can be automatically configured from physical storage capacity available at that site. Accordingly, there is no need to specially configure hardware or software at each site in order to implement a LUN that is compatible with a given copy set. Because each controller 105 can automatically increase the size or change the configuration of the LUN(s) it manages, the manual tasks involved to increase the size of a LUN do not grow geometrically or exponentially with the number of replicas in the copy set. Instead, the user simply manages the LUN as a single entity, and the implementation of changes in each replica is handled automatically.
  • It is apparent that the system of the present invention provides efficiency in that redundant components can operate simultaneously and in parallel to provide improved throughput as well as serving during failover to replace functionality of failed systems and connections. Because system components have few constraints on their functionality, the present invention allows great flexibility in implementing a particular system to emphasize any balance of, for example, low latency, high availability, disaster tolerance, and cost.
  • Although the invention has been described and illustrated with a certain degree of particularity, it is understood that the present disclosure has been made only by way of example, and that numerous changes in the combination and arrangement of parts can be resorted to by those skilled in the art without departing from the spirit and scope of the invention, as hereinafter claimed.

Claims (19)

1-17. (canceled)
18. A SAN storage controller comprising:
a data processor system configured to implement software processes;
at least one host port;
a pool of storage accessible through the data processor system;
processes executing in the processor to enable the at least one host port to couple to an external communication network;
processes executing in the processor for communicating data access requests with the at least one host port; and
processes executing in the processor for communicating data replication transactions with external storage controllers through the at least one host port.
19. The storage controller of claim 18 wherein the at least one port handles both data access requests and data replication transactions.
20. The storage controller of claim 18 further comprising at least two host ports.
21. The storage controller of claim 18 further comprising more than two host ports.
22. The storage controller of claim 18 further comprising processes within the controller causing the controller to register with a name server in the external communication network.
23. The storage controller of claim 22 wherein the name server stores a storage cell universally unique identification (UUID) value.
24. The storage controller of claim 23 further comprising processes for discovering external storage controllers coupled to the external communications network using the UUID registration information in the name server.
25. The storage controller of claim 18 wherein the at least one port comprises a fibre channel port.
26. The storage controller of claim 18 further comprising processes executing in the processor to virtualize the storage capacity.
27. A storage controller, comprising:
a first port adapted to provide a connection to a switch and configured to handle data transactions and data replication traffic;
a second port adapted to provide a connection to a switch and configured to handle data transactions and data replication traffic, wherein:
the first port serves as a primary port for a data replication operation with a first destination and a redundant port for a data replication operation with a second destination;
the second port serves as a primary port for a data replication operation with a second destination and a redundant port for a data replication operation with a first destination;
data transactions and data replication traffic managed by the first port are routed to the second port in when a data replication operation fails on the first controller; and
data transactions and data replication traffic managed by the second port are routed to the first port in when a data replication operation fails on the first controller.
28. The storage controller of claim 27, wherein at least one port is adapted to maintain a Fibre Channel Arbitrated Loop connection.
29. The storage controller of claim 27, wherein the storage controller comprises a unique identification number.
30. The storage controller of claim 27, wherein in response to a connection failure, the storage controller migrates data replication operations to a separate storage controller.
31. A data replication controller, comprising:
a data processor system configured to implement a software process;
a first port adapted to provide a connection to a switch and configured to handle data transactions and data replication traffic;
a second port adapted to provide a connection to a switch and configured to handle data transactions and data replication traffic;
a pool of storage accessible through the data replication controller;
a first software process executing in the data processor to enable the first host port to couple to an external communication network;
a second software process executing in the data processor to designate a source virtual disk in the pool of storage; and
a third process processes executing in the processor to create a destination virtual disk in the pool of storage.
32. The data replication controller of claim 31, wherein the first port and the second port communicate with a plurality of host systems to handle host data operations and with each other to handle data operations.
33. The data replication controller of claim 31, wherein at least one port is adapted to maintain a Fibre Channel Arbitrated Loop connection.
34. The data replication controller of claim 31, wherein the controller comprises a unique identification number.
35. The data replication controller of claim 31, wherein in response to a connection failure, the data replication controller migrates data replication operations to a separate controller.
US11/151,499 2002-03-26 2005-06-13 Flexible data replication mechanism Abandoned US20050243611A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US11/151,499 US20050243611A1 (en) 2002-03-26 2005-06-13 Flexible data replication mechanism

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US10/106,908 US6947981B2 (en) 2002-03-26 2002-03-26 Flexible data replication mechanism
US11/151,499 US20050243611A1 (en) 2002-03-26 2005-06-13 Flexible data replication mechanism

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
US10/106,908 Continuation US6947981B2 (en) 2002-03-26 2002-03-26 Flexible data replication mechanism

Publications (1)

Publication Number Publication Date
US20050243611A1 true US20050243611A1 (en) 2005-11-03

Family

ID=28452576

Family Applications (2)

Application Number Title Priority Date Filing Date
US10/106,908 Expired - Lifetime US6947981B2 (en) 2002-03-26 2002-03-26 Flexible data replication mechanism
US11/151,499 Abandoned US20050243611A1 (en) 2002-03-26 2005-06-13 Flexible data replication mechanism

Family Applications Before (1)

Application Number Title Priority Date Filing Date
US10/106,908 Expired - Lifetime US6947981B2 (en) 2002-03-26 2002-03-26 Flexible data replication mechanism

Country Status (1)

Country Link
US (2) US6947981B2 (en)

Cited By (34)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070073988A1 (en) * 2005-09-27 2007-03-29 Hitachi, Ltd. Data processing system, data management method and storage system
US20070208836A1 (en) * 2005-12-27 2007-09-06 Emc Corporation Presentation of virtual arrays using n-port ID virtualization
US20070263637A1 (en) * 2005-12-27 2007-11-15 Emc Corporation On-line data migration of a logical/virtual storage array
US7697554B1 (en) 2005-12-27 2010-04-13 Emc Corporation On-line data migration of a logical/virtual storage array by replacing virtual names
US7757059B1 (en) 2006-06-29 2010-07-13 Emc Corporation Virtual array non-disruptive management data migration
US20100251002A1 (en) * 2009-03-31 2010-09-30 Swaminathan Sivasubramanian Monitoring and Automated Recovery of Data Instances
US20100250748A1 (en) * 2009-03-31 2010-09-30 Swaminathan Sivasubramanian Monitoring and Automatic Scaling of Data Volumes
US20100250499A1 (en) * 2009-03-31 2010-09-30 Mcalister Grant Alexander Macdonald Cloning and Recovery of Data Volumes
US20100251242A1 (en) * 2009-03-31 2010-09-30 Swaminathan Sivasubramanian Control Service for Relational Data Management
US20110258136A1 (en) * 2010-04-20 2011-10-20 Accenture Global Services Gmbh Integration framework for enterprise content management systems
US8072987B1 (en) 2005-09-30 2011-12-06 Emc Corporation Full array non-disruptive data migration
US8074107B2 (en) 2009-10-26 2011-12-06 Amazon Technologies, Inc. Failover and recovery for replicated data instances
US8107467B1 (en) 2005-09-30 2012-01-31 Emc Corporation Full array non-disruptive failover
US8307003B1 (en) 2009-03-31 2012-11-06 Amazon Technologies, Inc. Self-service control environment
US8335765B2 (en) 2009-10-26 2012-12-18 Amazon Technologies, Inc. Provisioning and managing replicated data instances
US8452928B1 (en) 2006-06-29 2013-05-28 Emc Corporation Virtual array non-disruptive migration of extended storage functionality
US8533408B1 (en) 2006-06-29 2013-09-10 Emc Corporation Consolidating N-storage arrays into one storage array using virtual array non-disruptive data migration
US8539177B1 (en) 2006-06-29 2013-09-17 Emc Corporation Partitioning of a storage array into N-storage arrays using virtual array non-disruptive data migration
US8583861B1 (en) 2006-06-29 2013-11-12 Emc Corporation Presentation of management functionality of virtual arrays
US8589504B1 (en) 2006-06-29 2013-11-19 Emc Corporation Full array non-disruptive management data migration
US8676753B2 (en) 2009-10-26 2014-03-18 Amazon Technologies, Inc. Monitoring of replicated data instances
US20140172796A1 (en) * 2005-12-19 2014-06-19 Commvault Systems, Inc. Systems and methods for migrating components in a hierarchical storage network
US9063895B1 (en) * 2007-06-29 2015-06-23 Emc Corporation System and method of non-disruptive data migration between heterogeneous storage arrays
US9063896B1 (en) * 2007-06-29 2015-06-23 Emc Corporation System and method of non-disruptive data migration between virtual arrays of heterogeneous storage arrays
US9098211B1 (en) 2007-06-29 2015-08-04 Emc Corporation System and method of non-disruptive data migration between a full storage array and one or more virtual arrays
US9111220B2 (en) 2004-04-30 2015-08-18 Commvault Systems, Inc. Systems and methods for storage modeling and costing
US9135283B2 (en) 2009-10-07 2015-09-15 Amazon Technologies, Inc. Self-service configuration for data environment
US9164692B2 (en) 2004-04-30 2015-10-20 Commvault Systems, Inc. System and method for allocation of organizational resources
US9705888B2 (en) 2009-03-31 2017-07-11 Amazon Technologies, Inc. Managing security groups for data instances
US10176036B2 (en) 2015-10-29 2019-01-08 Commvault Systems, Inc. Monitoring, diagnosing, and repairing a management database in a data storage management system
US10275320B2 (en) 2015-06-26 2019-04-30 Commvault Systems, Inc. Incrementally accumulating in-process performance data and hierarchical reporting thereof for a data stream in a secondary copy operation
US10379988B2 (en) 2012-12-21 2019-08-13 Commvault Systems, Inc. Systems and methods for performance monitoring
US10831591B2 (en) 2018-01-11 2020-11-10 Commvault Systems, Inc. Remedial action based on maintaining process awareness in data storage management
US11449253B2 (en) 2018-12-14 2022-09-20 Commvault Systems, Inc. Disk usage growth prediction system

Families Citing this family (130)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5898830A (en) * 1996-10-17 1999-04-27 Network Engineering Software Firewall providing enhanced network security and user transparency
US6947981B2 (en) * 2002-03-26 2005-09-20 Hewlett-Packard Development Company, L.P. Flexible data replication mechanism
US8205009B2 (en) * 2002-04-25 2012-06-19 Emc Israel Development Center, Ltd. Apparatus for continuous compression of large volumes of data
US20040006587A1 (en) * 2002-07-02 2004-01-08 Dell Products L.P. Information handling system and method for clustering with internal cross coupled storage
JP3996010B2 (en) * 2002-08-01 2007-10-24 株式会社日立製作所 Storage network system, management apparatus, management method and program
KR20100072067A (en) * 2002-10-31 2010-06-29 링 테크노로지 엔터프라이즈, 엘엘씨 Methods and systems for a storage system
US7197662B2 (en) * 2002-10-31 2007-03-27 Ring Technology Enterprises, Llc Methods and systems for a storage system
US7707351B2 (en) * 2002-10-31 2010-04-27 Ring Technology Enterprises Of Texas, Llc Methods and systems for an identifier-based memory section
US6879526B2 (en) * 2002-10-31 2005-04-12 Ring Technology Enterprises Llc Methods and apparatus for improved memory access
US7415565B2 (en) * 2002-10-31 2008-08-19 Ring Technology Enterprises, Llc Methods and systems for a storage system with a program-controlled switch for routing data
JP2004178254A (en) * 2002-11-27 2004-06-24 Hitachi Ltd Information processing system, storage system, storage device controller and program
JP4278445B2 (en) * 2003-06-18 2009-06-17 株式会社日立製作所 Network system and switch
JP4406402B2 (en) * 2003-07-02 2010-01-27 聰 山竹 Database system
US20050097388A1 (en) * 2003-11-05 2005-05-05 Kris Land Data distributor
JP4516306B2 (en) 2003-11-28 2010-08-04 株式会社日立製作所 How to collect storage network performance information
WO2005086001A1 (en) * 2004-02-27 2005-09-15 Incipient, Inc. Distributed asynchronous ordered replication
US7149859B2 (en) * 2004-03-01 2006-12-12 Hitachi, Ltd. Method and apparatus for data migration with the efficient use of old assets
US20060182050A1 (en) * 2005-01-28 2006-08-17 Hewlett-Packard Development Company, L.P. Storage replication system with data tracking
US7519851B2 (en) * 2005-02-08 2009-04-14 Hitachi, Ltd. Apparatus for replicating volumes between heterogenous storage systems
US7895308B2 (en) * 2005-05-11 2011-02-22 Tindall Steven J Messaging system configurator
US7519859B2 (en) * 2005-08-30 2009-04-14 International Business Machines Corporation Fault recovery for transaction server
US8060713B1 (en) 2005-12-21 2011-11-15 Emc (Benelux) B.V., S.A.R.L. Consolidating snapshots in a continuous data protection system using journaling
US7774565B2 (en) * 2005-12-21 2010-08-10 Emc Israel Development Center, Ltd. Methods and apparatus for point in time data access and recovery
US7849361B2 (en) * 2005-12-22 2010-12-07 Emc Corporation Methods and apparatus for multiple point in time data access
US7577867B2 (en) * 2006-02-17 2009-08-18 Emc Corporation Cross tagging to data for consistent recovery
US7613750B2 (en) * 2006-05-29 2009-11-03 Microsoft Corporation Creating frequent application-consistent backups efficiently
US20070300033A1 (en) * 2006-06-21 2007-12-27 Hitachi, Ltd. System and method for reconfiguring continuous data protection system upon volume size change
US7627612B2 (en) * 2006-09-28 2009-12-01 Emc Israel Development Center, Ltd. Methods and apparatus for optimal journaling for continuous data replication
US7627687B2 (en) * 2006-09-28 2009-12-01 Emc Israel Development Center, Ltd. Methods and apparatus for managing data flow in a continuous data replication system having journaling
US20080114695A1 (en) 2006-11-10 2008-05-15 Semantic Components S.L. Process for implementing a method for the on-line sale of software product use licenses through a data network, and software component which allows carrying out said process
US20100250400A1 (en) * 2006-11-10 2010-09-30 Media Patents, S.L. Apparatus and methods for the sale of software products
US7702953B2 (en) * 2007-01-04 2010-04-20 International Business Machines Corporation Storage management in cascaded replication of data
WO2008122308A1 (en) * 2007-04-04 2008-10-16 Media Patents S.L. Process for the on-line distribution, through a data network, of digital files protected by intellectual property rights and computer readable medium containing a program for carrying out said process
US20110060688A1 (en) * 2007-11-23 2011-03-10 Media Patents, S.L. Apparatus and methods for the distribution of digital files
WO2009065526A1 (en) * 2007-11-23 2009-05-28 Media Patents S.L. A process for the on-line distribution of audiovisual contents with advertisements, advertisement management system, digital rights management system and audiovisual content player provided with said systems
US7860836B1 (en) 2007-12-26 2010-12-28 Emc (Benelux) B.V., S.A.R.L. Method and apparatus to recover data in a continuous data protection environment using a journal
US7958372B1 (en) 2007-12-26 2011-06-07 Emc (Benelux) B.V., S.A.R.L. Method and apparatus to convert a logical unit from a first encryption state to a second encryption state using a journal in a continuous data protection environment
US8041940B1 (en) 2007-12-26 2011-10-18 Emc Corporation Offloading encryption processing in a storage area network
US7840536B1 (en) 2007-12-26 2010-11-23 Emc (Benelux) B.V., S.A.R.L. Methods and apparatus for dynamic journal expansion
US8626936B2 (en) * 2008-01-23 2014-01-07 International Business Machines Corporation Protocol independent server replacement and replication in a storage area network
US9501542B1 (en) 2008-03-11 2016-11-22 Emc Corporation Methods and apparatus for volume synchronization
ES2326949B1 (en) * 2008-03-18 2010-07-14 Clarity Systems, S.L. PROCEDURE USED BY A STREAMING SERVER TO MAKE A TRANSMISSION OF A MULTIMEDIA FILE IN A DATA NETWORK.
US7984097B2 (en) * 2008-03-18 2011-07-19 Media Patents, S.L. Methods for transmitting multimedia files and advertisements
US8108634B1 (en) 2008-06-27 2012-01-31 Emc B.V., S.A.R.L. Replicating a thin logical unit
US7719443B1 (en) 2008-06-27 2010-05-18 Emc Corporation Compressing data in a continuous data protection environment
CN102084636B (en) * 2008-07-01 2014-01-15 国际商业机器公司 Storage area network configuration
US8060714B1 (en) 2008-09-26 2011-11-15 Emc (Benelux) B.V., S.A.R.L. Initializing volumes in a replication system
US7882286B1 (en) 2008-09-26 2011-02-01 EMC (Benelux)B.V., S.A.R.L. Synchronizing volumes for replication
US8122151B2 (en) * 2008-12-26 2012-02-21 Hitachi, Ltd. Storage system for optimally controlling a plurality of data transfer paths and method therefor
US9154532B2 (en) 2009-04-27 2015-10-06 Zaron Remote Llc Methods and apparatus for transmitting multimedia files in a data network
US20110023046A1 (en) * 2009-07-22 2011-01-27 Stephen Gold Mitigating resource usage during virtual storage replication
US8095684B2 (en) * 2009-09-15 2012-01-10 Symantec Corporation Intelligent device and media server selection for optimized backup image duplication
US8850142B2 (en) * 2009-09-15 2014-09-30 Hewlett-Packard Development Company, L.P. Enhanced virtual storage replication
US8996909B2 (en) * 2009-10-08 2015-03-31 Microsoft Corporation Modeling distribution and failover database connectivity behavior
US20110191749A1 (en) * 2010-01-29 2011-08-04 Martijn Van Liebergen System and method for generating enterprise applications
US9158653B2 (en) * 2010-03-21 2015-10-13 Hewlett-Packard Development Company, L.P. Determining impact of virtual storage backup jobs
US8392680B1 (en) 2010-03-30 2013-03-05 Emc International Company Accessing a volume in a distributed environment
US8332687B1 (en) 2010-06-23 2012-12-11 Emc Corporation Splitter used in a continuous data protection environment
US8370588B2 (en) * 2010-09-09 2013-02-05 Hitachi, Ltd. Computer system control method and computer system
US9009724B2 (en) 2010-09-24 2015-04-14 Hewlett-Packard Development Company, L.P. Load balancing data access in virtualized storage nodes
US8433869B1 (en) 2010-09-27 2013-04-30 Emc International Company Virtualized consistency group using an enhanced splitter
US8478955B1 (en) 2010-09-27 2013-07-02 Emc International Company Virtualized consistency group using more than one data protection appliance
US8694700B1 (en) 2010-09-29 2014-04-08 Emc Corporation Using I/O track information for continuous push with splitter for storage device
US8335771B1 (en) 2010-09-29 2012-12-18 Emc Corporation Storage array snapshots for logged access replication in a continuous data protection system
US8335761B1 (en) 2010-12-02 2012-12-18 Emc International Company Replicating in a multi-copy environment
US8468132B1 (en) 2010-12-28 2013-06-18 Amazon Technologies, Inc. Data replication framework
US9449065B1 (en) * 2010-12-28 2016-09-20 Amazon Technologies, Inc. Data replication framework
US8554762B1 (en) 2010-12-28 2013-10-08 Amazon Technologies, Inc. Data replication framework
US10198492B1 (en) 2010-12-28 2019-02-05 Amazon Technologies, Inc. Data replication framework
EP2487867B1 (en) * 2011-02-09 2014-01-01 Siemens Aktiengesellschaft Keep alive message monitoring
US8554951B2 (en) 2011-03-08 2013-10-08 Rackspace Us, Inc. Synchronization and ordering of multiple accessess in a distributed system
US8538926B2 (en) 2011-03-08 2013-09-17 Rackspace Us, Inc. Massively scalable object storage system for storing object replicas
US8510267B2 (en) 2011-03-08 2013-08-13 Rackspace Us, Inc. Synchronization of structured information repositories
US8712975B2 (en) * 2011-03-08 2014-04-29 Rackspace Us, Inc. Modification of an object replica
US9256605B1 (en) 2011-08-03 2016-02-09 Emc Corporation Reading and writing to an unexposed device
US8898112B1 (en) 2011-09-07 2014-11-25 Emc Corporation Write signature command
JP5899893B2 (en) 2011-12-19 2016-04-06 富士通株式会社 Storage device, command execution control method, and program
US9158714B2 (en) * 2012-01-31 2015-10-13 Symantec Corporation Method and system for multi-layer differential load balancing in tightly coupled clusters
US9015371B1 (en) * 2012-03-01 2015-04-21 Symantec Corporation Method to discover multiple paths to disk devices cluster wide
US9223659B1 (en) 2012-06-28 2015-12-29 Emc International Company Generating and accessing a virtual volume snapshot in a continuous data protection system
US10235145B1 (en) 2012-09-13 2019-03-19 Emc International Company Distributed scale-out replication
US9336094B1 (en) 2012-09-13 2016-05-10 Emc International Company Scaleout replication of an application
US9696939B1 (en) 2013-03-14 2017-07-04 EMC IP Holding Company LLC Replicating data using deduplication-based arrays using network-based replication
US8996460B1 (en) 2013-03-14 2015-03-31 Emc Corporation Accessing an image in a continuous data protection using deduplication-based storage
US9110914B1 (en) 2013-03-14 2015-08-18 Emc Corporation Continuous data protection using deduplication-based storage
US9383937B1 (en) 2013-03-14 2016-07-05 Emc Corporation Journal tiering in a continuous data protection system using deduplication-based storage
US9081842B1 (en) 2013-03-15 2015-07-14 Emc Corporation Synchronous and asymmetric asynchronous active-active-active data access
US9152339B1 (en) 2013-03-15 2015-10-06 Emc Corporation Synchronization of asymmetric active-active, asynchronously-protected storage
US9244997B1 (en) 2013-03-15 2016-01-26 Emc Corporation Asymmetric active-active access of asynchronously-protected data storage
US9069709B1 (en) 2013-06-24 2015-06-30 Emc International Company Dynamic granularity in data replication
US9087112B1 (en) 2013-06-24 2015-07-21 Emc International Company Consistency across snapshot shipping and continuous replication
US9146878B1 (en) 2013-06-25 2015-09-29 Emc Corporation Storage recovery from total cache loss using journal-based replication
US9367260B1 (en) 2013-12-13 2016-06-14 Emc Corporation Dynamic replication system
US9405765B1 (en) 2013-12-17 2016-08-02 Emc Corporation Replication of virtual machines
US9158630B1 (en) 2013-12-19 2015-10-13 Emc Corporation Testing integrity of replicated storage
US9189339B1 (en) 2014-03-28 2015-11-17 Emc Corporation Replication of a virtual distributed volume with virtual machine granualarity
US9558143B2 (en) * 2014-05-09 2017-01-31 Micron Technology, Inc. Interconnect systems and methods using hybrid memory cube links to send packetized data over different endpoints of a data handling device
US9274718B1 (en) 2014-06-20 2016-03-01 Emc Corporation Migration in replication system
US10082980B1 (en) 2014-06-20 2018-09-25 EMC IP Holding Company LLC Migration of snapshot in replication system using a log
US9619543B1 (en) 2014-06-23 2017-04-11 EMC IP Holding Company LLC Replicating in virtual desktop infrastructure
CN105447033B (en) 2014-08-28 2019-06-11 国际商业机器公司 The method and apparatus of initial copy are generated in duplication initialization
US10437783B1 (en) 2014-09-25 2019-10-08 EMC IP Holding Company LLC Recover storage array using remote deduplication device
US10101943B1 (en) 2014-09-25 2018-10-16 EMC IP Holding Company LLC Realigning data in replication system
US10324798B1 (en) 2014-09-25 2019-06-18 EMC IP Holding Company LLC Restoring active areas of a logical unit
US9529885B1 (en) 2014-09-29 2016-12-27 EMC IP Holding Company LLC Maintaining consistent point-in-time in asynchronous replication during virtual machine relocation
US9910621B1 (en) 2014-09-29 2018-03-06 EMC IP Holding Company LLC Backlogging I/O metadata utilizing counters to monitor write acknowledgements and no acknowledgements
US10496487B1 (en) 2014-12-03 2019-12-03 EMC IP Holding Company LLC Storing snapshot changes with snapshots
US9600377B1 (en) 2014-12-03 2017-03-21 EMC IP Holding Company LLC Providing data protection using point-in-time images from multiple types of storage devices
US9405481B1 (en) 2014-12-17 2016-08-02 Emc Corporation Replicating using volume multiplexing with consistency group file
US9632881B1 (en) 2015-03-24 2017-04-25 EMC IP Holding Company LLC Replication of a virtual distributed volume
US9411535B1 (en) 2015-03-27 2016-08-09 Emc Corporation Accessing multiple virtual devices
US10296419B1 (en) 2015-03-27 2019-05-21 EMC IP Holding Company LLC Accessing a virtual device using a kernel
US9678680B1 (en) 2015-03-30 2017-06-13 EMC IP Holding Company LLC Forming a protection domain in a storage architecture
US10853181B1 (en) 2015-06-29 2020-12-01 EMC IP Holding Company LLC Backing up volumes using fragment files
US10079883B2 (en) 2015-12-03 2018-09-18 International Business Machines Corporation Primary device selection at operating system initialization
US9684576B1 (en) 2015-12-21 2017-06-20 EMC IP Holding Company LLC Replication using a virtual distributed volume
US10067837B1 (en) 2015-12-28 2018-09-04 EMC IP Holding Company LLC Continuous data protection with cloud resources
US10235196B1 (en) 2015-12-28 2019-03-19 EMC IP Holding Company LLC Virtual machine joining or separating
US10133874B1 (en) 2015-12-28 2018-11-20 EMC IP Holding Company LLC Performing snapshot replication on a storage system not configured to support snapshot replication
US10152267B1 (en) 2016-03-30 2018-12-11 Emc Corporation Replication data pull
US10579282B1 (en) 2016-03-30 2020-03-03 EMC IP Holding Company LLC Distributed copy in multi-copy replication where offset and size of I/O requests to replication site is half offset and size of I/O request to production volume
US10235087B1 (en) 2016-03-30 2019-03-19 EMC IP Holding Company LLC Distributing journal data over multiple journals
US10146961B1 (en) 2016-09-23 2018-12-04 EMC IP Holding Company LLC Encrypting replication journals in a storage system
US10210073B1 (en) 2016-09-23 2019-02-19 EMC IP Holding Company, LLC Real time debugging of production replicated data with data obfuscation in a storage system
US10019194B1 (en) 2016-09-23 2018-07-10 EMC IP Holding Company LLC Eventually consistent synchronous data replication in a storage system
US10235090B1 (en) 2016-09-23 2019-03-19 EMC IP Holding Company LLC Validating replication copy consistency using a hash function in a storage system
US10235091B1 (en) 2016-09-23 2019-03-19 EMC IP Holding Company LLC Full sweep disk synchronization in a storage system
US10924455B2 (en) * 2017-03-22 2021-02-16 Dell Products L.P. Method and system for implementing a cluster-wide communication over a single port
CN108628549A (en) * 2018-04-26 2018-10-09 北京全路通信信号研究设计院集团有限公司 A kind of database management systems for railway dispatching center
US11531566B2 (en) * 2020-07-10 2022-12-20 Infineon Technologies Ag Safe and secure communication network message processing

Citations (93)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5682518A (en) * 1994-03-28 1997-10-28 Nec Corporation System for updating inactive system memory using dual port memory
US5692155A (en) * 1995-04-19 1997-11-25 International Business Machines Corporation Method and apparatus for suspending multiple duplex pairs during back up processing to insure storage devices remain synchronized in a sequence consistent order
US5870537A (en) * 1996-03-13 1999-02-09 International Business Machines Corporation Concurrent switch to shadowed device for storage controller and device errors
US5923876A (en) * 1995-08-24 1999-07-13 Compaq Computer Corp. Disk fault prediction system
US6148414A (en) * 1998-09-24 2000-11-14 Seek Systems, Inc. Methods and systems for implementing shared disk array management functions
US6161192A (en) * 1995-10-13 2000-12-12 Compaq Computer Corporation Raid array data storage system with storage device consistency bits and raidset consistency bits
US6170063B1 (en) * 1998-03-07 2001-01-02 Hewlett-Packard Company Method for performing atomic, concurrent read and write operations on multiple storage devices
US20010019920A1 (en) * 2000-01-25 2001-09-06 Framatome Connectors International Control unit of a valve mechanism
US6295578B1 (en) * 1999-04-09 2001-09-25 Compaq Computer Corporation Cascaded removable media data storage system
US20020019923A1 (en) * 2000-06-02 2002-02-14 Reuter James M. Distributed fine-grained enhancements for distributed table driven I/O mapping
US20020019863A1 (en) * 2000-06-02 2002-02-14 Reuter James M. Structure and process for distributing SCSI LUN semantics across parallel distributed components
US20020019908A1 (en) * 2000-06-02 2002-02-14 Reuter James M. System and method for managing virtual storage
US20020048284A1 (en) * 2000-02-18 2002-04-25 Moulton Gregory Hagan System and method for data protection with multidimensional parity
US6397293B2 (en) * 1998-06-23 2002-05-28 Hewlett-Packard Company Storage management system and auto-RAID transaction manager for coherent memory map across hot plug interface
US6487636B1 (en) * 2000-04-24 2002-11-26 Hewlett-Packard Co. Method and apparatus for mapping data in a heterogeneous disk array storage system
US6490122B1 (en) * 1998-08-28 2002-12-03 Hewlett-Packard Company System and method for providing power and control signals to a cartridge access device in a cartridge storage system
US6493656B1 (en) * 1999-02-26 2002-12-10 Compaq Computer Corporation, Inc. Drive error logging
US20020188800A1 (en) * 2001-05-15 2002-12-12 Tomaszewski Richard J. Self-mirroring high performance disk drive
US6505268B1 (en) * 1996-12-20 2003-01-07 Compaq Computer Corporation Data distribution in a disk array
US6523749B2 (en) * 2001-03-06 2003-02-25 Hewlett-Packard Company Apparatus and method for retrieving data cartridge information external to a media storage system
US20030051109A1 (en) * 2001-06-28 2003-03-13 Cochran Robert A. Method and system for providing logically consistent logical unit backup snapshots within one or more data storage devices
US20030056038A1 (en) * 2001-06-28 2003-03-20 Cochran Robert A. Method and system for providing advanced warning to a data stage device in order to decrease the time for a mirror split operation without starving host I/O request processsing
US20030063134A1 (en) * 2001-01-05 2003-04-03 Bob Lord System for displaying a hierarchical directory
US6546459B2 (en) * 2001-03-15 2003-04-08 Hewlett Packard Development Company, L. P. Redundant data storage systems and methods of operating a redundant data storage system
US20030074492A1 (en) * 2001-05-29 2003-04-17 Cochran Robert A. Method and system for efficient format, read, write, and initial copy processing involving sparse logical units
US20030079082A1 (en) * 2001-10-19 2003-04-24 Sicola Stephen J. Unified management system and method for multi-cabinet data storage complexes
US20030079014A1 (en) * 2001-10-22 2003-04-24 Lubbers Clark E. System and method for interfacing with virtual storage
US20030079156A1 (en) * 2001-10-19 2003-04-24 Sicola Stephen J. System and method for locating a failed storage device in a data storage system
US20030079083A1 (en) * 2001-10-22 2003-04-24 Lubbers Clark E. High performance multi-controller processing
US20030079102A1 (en) * 2001-06-01 2003-04-24 Lubbers Clark E. System and method for generating point in time storage copy
US20030084241A1 (en) * 2001-10-22 2003-05-01 Lubbers Clark E. System and method for atomizing storage
US6560673B2 (en) * 2001-01-31 2003-05-06 Hewlett Packard Development Company, L.P. Fibre channel upgrade path
US20030101318A1 (en) * 2001-11-26 2003-05-29 Hitachi, Ltd. Data copy method
US20030110237A1 (en) * 2001-12-06 2003-06-12 Hitachi, Ltd. Methods of migrating data between storage apparatuses
US6587962B1 (en) * 1999-10-20 2003-07-01 Hewlett-Packard Development Company, L.P. Write request protection upon failure in a multi-computer system
US20030126315A1 (en) * 2001-12-28 2003-07-03 Choon-Seng Tan Data storage network with host transparent failover controlled by host bus adapter
US20030126347A1 (en) * 2001-12-27 2003-07-03 Choon-Seng Tan Data array having redundancy messaging between array controllers over the host bus
US6594745B2 (en) * 2001-01-31 2003-07-15 Hewlett-Packard Development Company, L.P. Mirroring agent accessible to remote host computers, and accessing remote data-storage devices, via a communcations medium
US20030140191A1 (en) * 2002-01-24 2003-07-24 Mcgowen Michael E. System, method, and computer program product for on-line replacement of a host bus adapter
US6601187B1 (en) * 2000-03-31 2003-07-29 Hewlett-Packard Development Company, L. P. System for data replication using redundant pairs of storage controllers, fibre channel fabrics and links therebetween
US20030145130A1 (en) * 2002-01-31 2003-07-31 Schultz Stephen M. Array controller ROM cloning in redundant controllers
US20030145045A1 (en) * 2002-01-31 2003-07-31 Greg Pellegrino Storage aggregator for enhancing virtualization in data storage networks
US6606690B2 (en) * 2001-02-20 2003-08-12 Hewlett-Packard Development Company, L.P. System and method for accessing a storage area network as network attached storage
US6609145B1 (en) * 1995-10-13 2003-08-19 Hewlett-Packard Development Company, L.P. User selectable priority for disk array background operations
US20030170012A1 (en) * 2002-03-06 2003-09-11 Robert A. Cochran Method and system for reliable remote-mirror resynchronization in disk arrays and other mass storage devices
US20030177323A1 (en) * 2002-01-11 2003-09-18 Mathias Popp Remote mirrored disk pair resynchronization monitor
US6629273B1 (en) * 2000-01-24 2003-09-30 Hewlett-Packard Development Company, L.P. Detection of silent data corruption in a storage system
US6629108B2 (en) * 2001-03-09 2003-09-30 Hewlett-Packard Development Company, Lp. Method for insuring data integrity for mirrored independently accessible memory devices
US20030188114A1 (en) * 2002-03-26 2003-10-02 Clark Lubbers Data replication with virtualized volumes
US20030188119A1 (en) * 2002-03-26 2003-10-02 Clark Lubbers System and method for dynamically managing memory allocated to logging in a storage area network
US20030188085A1 (en) * 2002-04-02 2003-10-02 Hitachi, Ltd. Clustered storage system and its control method
US20030187947A1 (en) * 2002-03-26 2003-10-02 Clark Lubbers System and method for multi-destination merge in a storage area network
US20030188233A1 (en) * 2002-03-28 2003-10-02 Clark Lubbers System and method for automatic site failover in a storage area network
US20030188229A1 (en) * 2002-03-26 2003-10-02 Clark Lubbers System and method for managing data logging memory in a storage area network
US20030188153A1 (en) * 2002-04-02 2003-10-02 Demoff Jeff S. System and method for mirroring data using a server
US20030187847A1 (en) * 2002-03-26 2003-10-02 Clark Lubbers System and method for ensuring merge completion in a storage area network
US20030188218A1 (en) * 2002-03-26 2003-10-02 Clark Lubbers System and method for active-active data replication
US20030191919A1 (en) * 2002-04-08 2003-10-09 Hitachi, Ltd. Volume management method and apparatus
US20030191909A1 (en) * 2002-04-08 2003-10-09 Hitachi, Ltd. Computer system, storage and storage utilization and monitoring method
US20030196023A1 (en) * 1999-08-02 2003-10-16 Inostor Corporation Data redundancy methods and apparatus
US6643795B1 (en) * 2000-03-30 2003-11-04 Hewlett-Packard Development Company, L.P. Controller-based bi-directional remote copy system with storage site failover capability
US6647514B1 (en) * 2000-03-23 2003-11-11 Hewlett-Packard Development Company, L.P. Host I/O performance and availability of a storage array during rebuild by prioritizing I/O request
US20030212781A1 (en) * 2002-05-08 2003-11-13 Hitachi, Ltd. Network topology management system, management apparatus, management method, management program, and storage media that records management program
US20030229651A1 (en) * 2002-03-22 2003-12-11 Hitachi, Ltd. Snapshot acquisition method, storage system
US6681308B1 (en) * 2001-11-16 2004-01-20 Hewlett-Packard Development Company, L.P. Method for automatically converting block size and formatting backend fiber channel discs in an auto inclusive storage array environment
US20040019740A1 (en) * 2002-07-25 2004-01-29 Hitachi, Ltd. Destaging method for storage apparatus system, and disk control apparatus, storage apparatus system and program
US20040022546A1 (en) * 2002-07-31 2004-02-05 Cochran Robert A. Method and apparatus for compacting data in a communication network
US20040024838A1 (en) * 2002-07-31 2004-02-05 Cochran Robert A. Intelligent data tunnels multiplexed within communications media directly interconnecting two or more multi-logical-unit-mass-storage devices
US20040024961A1 (en) * 2002-07-31 2004-02-05 Cochran Robert A. Immediately available, statically allocated, full-logical-unit copy with a transient, snapshot-copy-like intermediate stage
US20040030846A1 (en) * 2002-08-06 2004-02-12 Philippe Armangau Data storage system having meta bit maps for indicating whether data blocks are invalid in snapshot copies
US20040030727A1 (en) * 2002-08-06 2004-02-12 Philippe Armangau Organization of multiple snapshot copies in a data storage system
US20040049634A1 (en) * 2001-04-17 2004-03-11 Cochran Robert A. Unified data sets distributed over multiple I/O-device arrays
US6708285B2 (en) * 2001-03-15 2004-03-16 Hewlett-Packard Development Company, L.P. Redundant controller data storage system having system and method for handling controller resets
US6715101B2 (en) * 2001-03-15 2004-03-30 Hewlett-Packard Development Company, L.P. Redundant controller data storage system having an on-line controller removal system and method
US6718434B2 (en) * 2001-05-31 2004-04-06 Hewlett-Packard Development Company, L.P. Method and apparatus for assigning raid levels
US6721902B1 (en) * 2000-10-12 2004-04-13 Hewlett-Packard Development Company, L.P. Method and system for providing LUN-based backup reliability via LUN-based locking
US6725393B1 (en) * 2000-11-06 2004-04-20 Hewlett-Packard Development Company, L.P. System, machine, and method for maintenance of mirrored datasets through surrogate writes during storage-area network transients
US20040078641A1 (en) * 2002-09-23 2004-04-22 Hewlett-Packard Company Operating system-independent file restore from disk image
US20040078638A1 (en) * 2002-07-31 2004-04-22 Cochran Robert A. Method and system for preventing data loss within disk-array pairs supporting mirrored logical units
US6742020B1 (en) * 2000-06-08 2004-05-25 Hewlett-Packard Development Company, L.P. System and method for managing data flow and measuring service in a storage network
US6763409B1 (en) * 2001-01-31 2004-07-13 Hewlett-Packard Development Company, L.P. Switch-on-the-fly GBIC disk channel adapter and disk channel system
US20040168034A1 (en) * 2003-02-26 2004-08-26 Hitachi, Ltd. Storage apparatus and its management method
US6795904B1 (en) * 2002-03-28 2004-09-21 Hewlett-Packard Development Company, L.P. System and method for improving performance of a data backup operation
US6802023B2 (en) * 2001-03-15 2004-10-05 Hewlett-Packard Development Company, L.P. Redundant controller data storage system having hot insertion system and method
US6807605B2 (en) * 2002-10-03 2004-10-19 Hewlett-Packard Development Company, L.P. Managing a data storage array, a data storage system, and a raid controller
US20040215602A1 (en) * 2003-04-23 2004-10-28 Hewlett-Packard Development Company, L.P. Method and system for distributed remote resources
US6817522B2 (en) * 2003-01-24 2004-11-16 Hewlett-Packard Development Company, L.P. System and method for distributed storage management
US20040230859A1 (en) * 2003-05-15 2004-11-18 Hewlett-Packard Development Company, L.P. Disaster recovery system with cascaded resynchronization
US6823453B1 (en) * 2000-10-06 2004-11-23 Hewlett-Packard Development Company, L.P. Apparatus and method for implementing spoofing-and replay-attack-resistant virtual zones on storage area networks
US6839824B2 (en) * 2001-12-28 2005-01-04 Hewlett-Packard Development Company, L.P. System and method for partitioning a storage area network associated data library employing element addresses
US6842833B1 (en) * 1998-06-30 2005-01-11 Hewlett-Packard Development Company, L.P. Computer system and method for transferring data between multiple peer-level storage units
US6845403B2 (en) * 2001-10-31 2005-01-18 Hewlett-Packard Development Company, L.P. System and method for storage virtualization
US6947981B2 (en) * 2002-03-26 2005-09-20 Hewlett-Packard Development Company, L.P. Flexible data replication mechanism

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5666514A (en) * 1994-07-01 1997-09-09 Board Of Trustees Of The Leland Stanford Junior University Cache memory containing extra status bits to indicate memory regions where logging of data should occur
US5533190A (en) * 1994-12-21 1996-07-02 At&T Global Information Solutions Company Method for maintaining parity-data consistency in a disk array
US5721918A (en) * 1996-02-06 1998-02-24 Telefonaktiebolaget Lm Ericsson Method and system for fast recovery of a primary store database using selective recovery by data type
US6041423A (en) * 1996-11-08 2000-03-21 Oracle Corporation Method and apparatus for using undo/redo logging to perform asynchronous updates of parity and data pages in a redundant array data storage environment
US6148368A (en) * 1997-07-31 2000-11-14 Lsi Logic Corporation Method for accelerating disk array write operations using segmented cache memory and data logging
JP3504138B2 (en) * 1998-03-20 2004-03-08 富士通株式会社 Transmission device log recording method
US6185663B1 (en) * 1998-06-15 2001-02-06 Compaq Computer Corporation Computer method and apparatus for file system block allocation with multiple redo
US6370605B1 (en) * 1999-03-04 2002-04-09 Sun Microsystems, Inc. Switch based scalable performance storage architecture
US6629264B1 (en) * 2000-03-30 2003-09-30 Hewlett-Packard Development Company, L.P. Controller-based remote copy system with logical unit grouping
US20020191649A1 (en) * 2001-06-13 2002-12-19 Woodring Sherrie L. Port mirroring in channel directors and switches
US8549048B2 (en) * 2001-12-19 2013-10-01 Emc Corporation Workflow database for scalable storage service
US7886298B2 (en) * 2002-03-26 2011-02-08 Hewlett-Packard Development Company, L.P. Data transfer protocol for data replication between multiple pairs of storage controllers on a san fabric
US7542986B2 (en) * 2002-03-26 2009-06-02 Hewlett-Packard Development Company, L.P. System and method for maintaining order for a replicated multi-unit I/O stream

Patent Citations (99)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5682518A (en) * 1994-03-28 1997-10-28 Nec Corporation System for updating inactive system memory using dual port memory
US5692155A (en) * 1995-04-19 1997-11-25 International Business Machines Corporation Method and apparatus for suspending multiple duplex pairs during back up processing to insure storage devices remain synchronized in a sequence consistent order
US5923876A (en) * 1995-08-24 1999-07-13 Compaq Computer Corp. Disk fault prediction system
US6161192A (en) * 1995-10-13 2000-12-12 Compaq Computer Corporation Raid array data storage system with storage device consistency bits and raidset consistency bits
US6609145B1 (en) * 1995-10-13 2003-08-19 Hewlett-Packard Development Company, L.P. User selectable priority for disk array background operations
US5870537A (en) * 1996-03-13 1999-02-09 International Business Machines Corporation Concurrent switch to shadowed device for storage controller and device errors
US6505268B1 (en) * 1996-12-20 2003-01-07 Compaq Computer Corporation Data distribution in a disk array
US6170063B1 (en) * 1998-03-07 2001-01-02 Hewlett-Packard Company Method for performing atomic, concurrent read and write operations on multiple storage devices
US6397293B2 (en) * 1998-06-23 2002-05-28 Hewlett-Packard Company Storage management system and auto-RAID transaction manager for coherent memory map across hot plug interface
US6842833B1 (en) * 1998-06-30 2005-01-11 Hewlett-Packard Development Company, L.P. Computer system and method for transferring data between multiple peer-level storage units
US6490122B1 (en) * 1998-08-28 2002-12-03 Hewlett-Packard Company System and method for providing power and control signals to a cartridge access device in a cartridge storage system
US6148414A (en) * 1998-09-24 2000-11-14 Seek Systems, Inc. Methods and systems for implementing shared disk array management functions
US6493656B1 (en) * 1999-02-26 2002-12-10 Compaq Computer Corporation, Inc. Drive error logging
US6295578B1 (en) * 1999-04-09 2001-09-25 Compaq Computer Corporation Cascaded removable media data storage system
US20030196023A1 (en) * 1999-08-02 2003-10-16 Inostor Corporation Data redundancy methods and apparatus
US6587962B1 (en) * 1999-10-20 2003-07-01 Hewlett-Packard Development Company, L.P. Write request protection upon failure in a multi-computer system
US6629273B1 (en) * 2000-01-24 2003-09-30 Hewlett-Packard Development Company, L.P. Detection of silent data corruption in a storage system
US20010019920A1 (en) * 2000-01-25 2001-09-06 Framatome Connectors International Control unit of a valve mechanism
US20020048284A1 (en) * 2000-02-18 2002-04-25 Moulton Gregory Hagan System and method for data protection with multidimensional parity
US6647514B1 (en) * 2000-03-23 2003-11-11 Hewlett-Packard Development Company, L.P. Host I/O performance and availability of a storage array during rebuild by prioritizing I/O request
US6643795B1 (en) * 2000-03-30 2003-11-04 Hewlett-Packard Development Company, L.P. Controller-based bi-directional remote copy system with storage site failover capability
US6601187B1 (en) * 2000-03-31 2003-07-29 Hewlett-Packard Development Company, L. P. System for data replication using redundant pairs of storage controllers, fibre channel fabrics and links therebetween
US6487636B1 (en) * 2000-04-24 2002-11-26 Hewlett-Packard Co. Method and apparatus for mapping data in a heterogeneous disk array storage system
US6775790B2 (en) * 2000-06-02 2004-08-10 Hewlett-Packard Development Company, L.P. Distributed fine-grained enhancements for distributed table driven I/O mapping
US20020019863A1 (en) * 2000-06-02 2002-02-14 Reuter James M. Structure and process for distributing SCSI LUN semantics across parallel distributed components
US6772231B2 (en) * 2000-06-02 2004-08-03 Hewlett-Packard Development Company, L.P. Structure and process for distributing SCSI LUN semantics across parallel distributed components
US20020019923A1 (en) * 2000-06-02 2002-02-14 Reuter James M. Distributed fine-grained enhancements for distributed table driven I/O mapping
US6745207B2 (en) * 2000-06-02 2004-06-01 Hewlett-Packard Development Company, L.P. System and method for managing virtual storage
US20020019922A1 (en) * 2000-06-02 2002-02-14 Reuter James M. Data migration using parallel, distributed table driven I/O mapping
US20020019908A1 (en) * 2000-06-02 2002-02-14 Reuter James M. System and method for managing virtual storage
US6718404B2 (en) * 2000-06-02 2004-04-06 Hewlett-Packard Development Company, L.P. Data migration using parallel, distributed table driven I/O mapping
US6742020B1 (en) * 2000-06-08 2004-05-25 Hewlett-Packard Development Company, L.P. System and method for managing data flow and measuring service in a storage network
US6823453B1 (en) * 2000-10-06 2004-11-23 Hewlett-Packard Development Company, L.P. Apparatus and method for implementing spoofing-and replay-attack-resistant virtual zones on storage area networks
US6721902B1 (en) * 2000-10-12 2004-04-13 Hewlett-Packard Development Company, L.P. Method and system for providing LUN-based backup reliability via LUN-based locking
US6725393B1 (en) * 2000-11-06 2004-04-20 Hewlett-Packard Development Company, L.P. System, machine, and method for maintenance of mirrored datasets through surrogate writes during storage-area network transients
US20030063134A1 (en) * 2001-01-05 2003-04-03 Bob Lord System for displaying a hierarchical directory
US6560673B2 (en) * 2001-01-31 2003-05-06 Hewlett Packard Development Company, L.P. Fibre channel upgrade path
US6594745B2 (en) * 2001-01-31 2003-07-15 Hewlett-Packard Development Company, L.P. Mirroring agent accessible to remote host computers, and accessing remote data-storage devices, via a communcations medium
US6763409B1 (en) * 2001-01-31 2004-07-13 Hewlett-Packard Development Company, L.P. Switch-on-the-fly GBIC disk channel adapter and disk channel system
US6606690B2 (en) * 2001-02-20 2003-08-12 Hewlett-Packard Development Company, L.P. System and method for accessing a storage area network as network attached storage
US6523749B2 (en) * 2001-03-06 2003-02-25 Hewlett-Packard Company Apparatus and method for retrieving data cartridge information external to a media storage system
US6629108B2 (en) * 2001-03-09 2003-09-30 Hewlett-Packard Development Company, Lp. Method for insuring data integrity for mirrored independently accessible memory devices
US6546459B2 (en) * 2001-03-15 2003-04-08 Hewlett Packard Development Company, L. P. Redundant data storage systems and methods of operating a redundant data storage system
US6708285B2 (en) * 2001-03-15 2004-03-16 Hewlett-Packard Development Company, L.P. Redundant controller data storage system having system and method for handling controller resets
US6715101B2 (en) * 2001-03-15 2004-03-30 Hewlett-Packard Development Company, L.P. Redundant controller data storage system having an on-line controller removal system and method
US6802023B2 (en) * 2001-03-15 2004-10-05 Hewlett-Packard Development Company, L.P. Redundant controller data storage system having hot insertion system and method
US20040049634A1 (en) * 2001-04-17 2004-03-11 Cochran Robert A. Unified data sets distributed over multiple I/O-device arrays
US20020188800A1 (en) * 2001-05-15 2002-12-12 Tomaszewski Richard J. Self-mirroring high performance disk drive
US20030074492A1 (en) * 2001-05-29 2003-04-17 Cochran Robert A. Method and system for efficient format, read, write, and initial copy processing involving sparse logical units
US6718434B2 (en) * 2001-05-31 2004-04-06 Hewlett-Packard Development Company, L.P. Method and apparatus for assigning raid levels
US20030079102A1 (en) * 2001-06-01 2003-04-24 Lubbers Clark E. System and method for generating point in time storage copy
US20030056038A1 (en) * 2001-06-28 2003-03-20 Cochran Robert A. Method and system for providing advanced warning to a data stage device in order to decrease the time for a mirror split operation without starving host I/O request processsing
US20040128404A1 (en) * 2001-06-28 2004-07-01 Cochran Robert A. Method and system for providing advanced warning to a data stage device in order to decrease the time for a mirror split operation without starving host I/O request processing
US20030051109A1 (en) * 2001-06-28 2003-03-13 Cochran Robert A. Method and system for providing logically consistent logical unit backup snapshots within one or more data storage devices
US20030079156A1 (en) * 2001-10-19 2003-04-24 Sicola Stephen J. System and method for locating a failed storage device in a data storage system
US20030079082A1 (en) * 2001-10-19 2003-04-24 Sicola Stephen J. Unified management system and method for multi-cabinet data storage complexes
US20030079014A1 (en) * 2001-10-22 2003-04-24 Lubbers Clark E. System and method for interfacing with virtual storage
US20030084241A1 (en) * 2001-10-22 2003-05-01 Lubbers Clark E. System and method for atomizing storage
US20030079083A1 (en) * 2001-10-22 2003-04-24 Lubbers Clark E. High performance multi-controller processing
US6845403B2 (en) * 2001-10-31 2005-01-18 Hewlett-Packard Development Company, L.P. System and method for storage virtualization
US6681308B1 (en) * 2001-11-16 2004-01-20 Hewlett-Packard Development Company, L.P. Method for automatically converting block size and formatting backend fiber channel discs in an auto inclusive storage array environment
US20030101318A1 (en) * 2001-11-26 2003-05-29 Hitachi, Ltd. Data copy method
US20030110237A1 (en) * 2001-12-06 2003-06-12 Hitachi, Ltd. Methods of migrating data between storage apparatuses
US20030126347A1 (en) * 2001-12-27 2003-07-03 Choon-Seng Tan Data array having redundancy messaging between array controllers over the host bus
US20030126315A1 (en) * 2001-12-28 2003-07-03 Choon-Seng Tan Data storage network with host transparent failover controlled by host bus adapter
US6839824B2 (en) * 2001-12-28 2005-01-04 Hewlett-Packard Development Company, L.P. System and method for partitioning a storage area network associated data library employing element addresses
US20030177323A1 (en) * 2002-01-11 2003-09-18 Mathias Popp Remote mirrored disk pair resynchronization monitor
US20030140191A1 (en) * 2002-01-24 2003-07-24 Mcgowen Michael E. System, method, and computer program product for on-line replacement of a host bus adapter
US20030145045A1 (en) * 2002-01-31 2003-07-31 Greg Pellegrino Storage aggregator for enhancing virtualization in data storage networks
US20030145130A1 (en) * 2002-01-31 2003-07-31 Schultz Stephen M. Array controller ROM cloning in redundant controllers
US20030170012A1 (en) * 2002-03-06 2003-09-11 Robert A. Cochran Method and system for reliable remote-mirror resynchronization in disk arrays and other mass storage devices
US20030229651A1 (en) * 2002-03-22 2003-12-11 Hitachi, Ltd. Snapshot acquisition method, storage system
US20030188218A1 (en) * 2002-03-26 2003-10-02 Clark Lubbers System and method for active-active data replication
US20030187947A1 (en) * 2002-03-26 2003-10-02 Clark Lubbers System and method for multi-destination merge in a storage area network
US20030188119A1 (en) * 2002-03-26 2003-10-02 Clark Lubbers System and method for dynamically managing memory allocated to logging in a storage area network
US20030188114A1 (en) * 2002-03-26 2003-10-02 Clark Lubbers Data replication with virtualized volumes
US6947981B2 (en) * 2002-03-26 2005-09-20 Hewlett-Packard Development Company, L.P. Flexible data replication mechanism
US20030188229A1 (en) * 2002-03-26 2003-10-02 Clark Lubbers System and method for managing data logging memory in a storage area network
US20030187847A1 (en) * 2002-03-26 2003-10-02 Clark Lubbers System and method for ensuring merge completion in a storage area network
US20030188233A1 (en) * 2002-03-28 2003-10-02 Clark Lubbers System and method for automatic site failover in a storage area network
US6795904B1 (en) * 2002-03-28 2004-09-21 Hewlett-Packard Development Company, L.P. System and method for improving performance of a data backup operation
US20030188085A1 (en) * 2002-04-02 2003-10-02 Hitachi, Ltd. Clustered storage system and its control method
US20030188153A1 (en) * 2002-04-02 2003-10-02 Demoff Jeff S. System and method for mirroring data using a server
US20030191919A1 (en) * 2002-04-08 2003-10-09 Hitachi, Ltd. Volume management method and apparatus
US20030191909A1 (en) * 2002-04-08 2003-10-09 Hitachi, Ltd. Computer system, storage and storage utilization and monitoring method
US20030212781A1 (en) * 2002-05-08 2003-11-13 Hitachi, Ltd. Network topology management system, management apparatus, management method, management program, and storage media that records management program
US20040019740A1 (en) * 2002-07-25 2004-01-29 Hitachi, Ltd. Destaging method for storage apparatus system, and disk control apparatus, storage apparatus system and program
US20040078638A1 (en) * 2002-07-31 2004-04-22 Cochran Robert A. Method and system for preventing data loss within disk-array pairs supporting mirrored logical units
US20040022546A1 (en) * 2002-07-31 2004-02-05 Cochran Robert A. Method and apparatus for compacting data in a communication network
US20040024838A1 (en) * 2002-07-31 2004-02-05 Cochran Robert A. Intelligent data tunnels multiplexed within communications media directly interconnecting two or more multi-logical-unit-mass-storage devices
US20040024961A1 (en) * 2002-07-31 2004-02-05 Cochran Robert A. Immediately available, statically allocated, full-logical-unit copy with a transient, snapshot-copy-like intermediate stage
US20040030727A1 (en) * 2002-08-06 2004-02-12 Philippe Armangau Organization of multiple snapshot copies in a data storage system
US20040030846A1 (en) * 2002-08-06 2004-02-12 Philippe Armangau Data storage system having meta bit maps for indicating whether data blocks are invalid in snapshot copies
US20040078641A1 (en) * 2002-09-23 2004-04-22 Hewlett-Packard Company Operating system-independent file restore from disk image
US6807605B2 (en) * 2002-10-03 2004-10-19 Hewlett-Packard Development Company, L.P. Managing a data storage array, a data storage system, and a raid controller
US6817522B2 (en) * 2003-01-24 2004-11-16 Hewlett-Packard Development Company, L.P. System and method for distributed storage management
US20040168034A1 (en) * 2003-02-26 2004-08-26 Hitachi, Ltd. Storage apparatus and its management method
US20040215602A1 (en) * 2003-04-23 2004-10-28 Hewlett-Packard Development Company, L.P. Method and system for distributed remote resources
US20040230859A1 (en) * 2003-05-15 2004-11-18 Hewlett-Packard Development Company, L.P. Disaster recovery system with cascaded resynchronization

Cited By (89)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11287974B2 (en) 2004-04-30 2022-03-29 Commvault Systems, Inc. Systems and methods for storage modeling and costing
US9111220B2 (en) 2004-04-30 2015-08-18 Commvault Systems, Inc. Systems and methods for storage modeling and costing
US9164692B2 (en) 2004-04-30 2015-10-20 Commvault Systems, Inc. System and method for allocation of organizational resources
US9405471B2 (en) 2004-04-30 2016-08-02 Commvault Systems, Inc. Systems and methods for storage modeling and costing
US10282113B2 (en) 2004-04-30 2019-05-07 Commvault Systems, Inc. Systems and methods for providing a unified view of primary and secondary storage resources
US10901615B2 (en) 2004-04-30 2021-01-26 Commvault Systems, Inc. Systems and methods for storage modeling and costing
US8127094B2 (en) 2005-09-27 2012-02-28 Hitachi, Ltd. Data migration and copying in a storage system with dynamically expansible volumes
US7464232B2 (en) * 2005-09-27 2008-12-09 Hitachi, Ltd. Data migration and copying in a storage system with dynamically expansible volumes
US20070073988A1 (en) * 2005-09-27 2007-03-29 Hitachi, Ltd. Data processing system, data management method and storage system
US20090077331A1 (en) * 2005-09-27 2009-03-19 Hitachi, Ltd. Data migration and copying in a storage system with dynamically expansible volumes
US7945748B2 (en) 2005-09-27 2011-05-17 Hitachi, Ltd. Data migration and copying in a storage system with dynamically expansible volumes
US20110202742A1 (en) * 2005-09-27 2011-08-18 Hitachi, Ltd. Data migration and copying in a storage system with dynamically expansible volumes
US8312235B2 (en) 2005-09-27 2012-11-13 Hitachi, Ltd. Data migration in a storage system with dynamically expansible volumes
US8072987B1 (en) 2005-09-30 2011-12-06 Emc Corporation Full array non-disruptive data migration
US8107467B1 (en) 2005-09-30 2012-01-31 Emc Corporation Full array non-disruptive failover
US9916111B2 (en) * 2005-12-19 2018-03-13 Commvault Systems, Inc. Systems and methods for migrating components in a hierarchical storage network
US20150339197A1 (en) * 2005-12-19 2015-11-26 Commvault Systems, Inc. Systems and methods for migrating components in a hierarchical storage network
US9152685B2 (en) * 2005-12-19 2015-10-06 Commvault Systems, Inc. Systems and methods for migrating components in a hierarchical storage network
US20140172796A1 (en) * 2005-12-19 2014-06-19 Commvault Systems, Inc. Systems and methods for migrating components in a hierarchical storage network
US9448892B2 (en) * 2005-12-19 2016-09-20 Commvault Systems, Inc. Systems and methods for migrating components in a hierarchical storage network
US10133507B2 (en) * 2005-12-19 2018-11-20 Commvault Systems, Inc Systems and methods for migrating components in a hierarchical storage network
US20160306589A1 (en) * 2005-12-19 2016-10-20 Commvault Systems, Inc. Systems and methods for migrating components in a hierarchical storage network
US11132139B2 (en) * 2005-12-19 2021-09-28 Commvault Systems, Inc. Systems and methods for migrating components in a hierarchical storage network
US7697554B1 (en) 2005-12-27 2010-04-13 Emc Corporation On-line data migration of a logical/virtual storage array by replacing virtual names
US9348530B2 (en) 2005-12-27 2016-05-24 Emc Corporation Presentation of virtual arrays using n-port ID virtualization
US20070263637A1 (en) * 2005-12-27 2007-11-15 Emc Corporation On-line data migration of a logical/virtual storage array
US20070208836A1 (en) * 2005-12-27 2007-09-06 Emc Corporation Presentation of virtual arrays using n-port ID virtualization
US8533408B1 (en) 2006-06-29 2013-09-10 Emc Corporation Consolidating N-storage arrays into one storage array using virtual array non-disruptive data migration
US8539177B1 (en) 2006-06-29 2013-09-17 Emc Corporation Partitioning of a storage array into N-storage arrays using virtual array non-disruptive data migration
US8583861B1 (en) 2006-06-29 2013-11-12 Emc Corporation Presentation of management functionality of virtual arrays
US8589504B1 (en) 2006-06-29 2013-11-19 Emc Corporation Full array non-disruptive management data migration
US8452928B1 (en) 2006-06-29 2013-05-28 Emc Corporation Virtual array non-disruptive migration of extended storage functionality
US7757059B1 (en) 2006-06-29 2010-07-13 Emc Corporation Virtual array non-disruptive management data migration
US9098211B1 (en) 2007-06-29 2015-08-04 Emc Corporation System and method of non-disruptive data migration between a full storage array and one or more virtual arrays
US9063896B1 (en) * 2007-06-29 2015-06-23 Emc Corporation System and method of non-disruptive data migration between virtual arrays of heterogeneous storage arrays
US9063895B1 (en) * 2007-06-29 2015-06-23 Emc Corporation System and method of non-disruptive data migration between heterogeneous storage arrays
US8332365B2 (en) * 2009-03-31 2012-12-11 Amazon Technologies, Inc. Cloning and recovery of data volumes
US20100250748A1 (en) * 2009-03-31 2010-09-30 Swaminathan Sivasubramanian Monitoring and Automatic Scaling of Data Volumes
US8713060B2 (en) 2009-03-31 2014-04-29 Amazon Technologies, Inc. Control service for relational data management
US8706764B2 (en) 2009-03-31 2014-04-22 Amazon Technologies, Inc. Control service for relational data management
US11914486B2 (en) * 2009-03-31 2024-02-27 Amazon Technologies, Inc. Cloning and recovery of data volumes
US20100251242A1 (en) * 2009-03-31 2010-09-30 Swaminathan Sivasubramanian Control Service for Relational Data Management
US8631283B1 (en) 2009-03-31 2014-01-14 Amazon Technologies, Inc. Monitoring and automated recovery of data instances
US20230070982A1 (en) * 2009-03-31 2023-03-09 Amazon Technologies, Inc. Cloning and recovery of data volumes
US8612396B1 (en) 2009-03-31 2013-12-17 Amazon Technologies, Inc. Cloning and recovery of data volumes
US9207984B2 (en) 2009-03-31 2015-12-08 Amazon Technologies, Inc. Monitoring and automatic scaling of data volumes
US9218245B1 (en) * 2009-03-31 2015-12-22 Amazon Technologies, Inc. Cloning and recovery of data volumes
US11550630B2 (en) 2009-03-31 2023-01-10 Amazon Technologies, Inc. Monitoring and automatic scaling of data volumes
US11385969B2 (en) * 2009-03-31 2022-07-12 Amazon Technologies, Inc. Cloning and recovery of data volumes
US10761975B2 (en) 2009-03-31 2020-09-01 Amazon Technologies, Inc. Control service for data management
US11379332B2 (en) 2009-03-31 2022-07-05 Amazon Technologies, Inc. Control service for data management
US8307003B1 (en) 2009-03-31 2012-11-06 Amazon Technologies, Inc. Self-service control environment
US20100251002A1 (en) * 2009-03-31 2010-09-30 Swaminathan Sivasubramanian Monitoring and Automated Recovery of Data Instances
US9705888B2 (en) 2009-03-31 2017-07-11 Amazon Technologies, Inc. Managing security groups for data instances
US11132227B2 (en) 2009-03-31 2021-09-28 Amazon Technologies, Inc. Monitoring and automatic scaling of data volumes
US10282231B1 (en) 2009-03-31 2019-05-07 Amazon Technologies, Inc. Monitoring and automatic scaling of data volumes
US8060792B2 (en) 2009-03-31 2011-11-15 Amazon Technologies, Inc. Monitoring and automated recovery of data instances
US10127149B2 (en) 2009-03-31 2018-11-13 Amazon Technologies, Inc. Control service for data management
US20100250499A1 (en) * 2009-03-31 2010-09-30 Mcalister Grant Alexander Macdonald Cloning and Recovery of Data Volumes
US10162715B1 (en) * 2009-03-31 2018-12-25 Amazon Technologies, Inc. Cloning and recovery of data volumes
US8713061B1 (en) 2009-04-03 2014-04-29 Amazon Technologies, Inc. Self-service administration of a database
US10977226B2 (en) 2009-10-07 2021-04-13 Amazon Technologies, Inc. Self-service configuration for data environment
US9135283B2 (en) 2009-10-07 2015-09-15 Amazon Technologies, Inc. Self-service configuration for data environment
US8595547B1 (en) 2009-10-26 2013-11-26 Amazon Technologies, Inc. Failover and recovery for replicated data instances
US9817727B2 (en) 2009-10-26 2017-11-14 Amazon Technologies, Inc. Failover and recovery for replicated data instances
US9298728B2 (en) 2009-10-26 2016-03-29 Amazon Technologies, Inc. Failover and recovery for replicated data instances
US8676753B2 (en) 2009-10-26 2014-03-18 Amazon Technologies, Inc. Monitoring of replicated data instances
US11907254B2 (en) 2009-10-26 2024-02-20 Amazon Technologies, Inc. Provisioning and managing replicated data instances
US11714726B2 (en) 2009-10-26 2023-08-01 Amazon Technologies, Inc. Failover and recovery for replicated data instances
US10860439B2 (en) 2009-10-26 2020-12-08 Amazon Technologies, Inc. Failover and recovery for replicated data instances
US9336292B2 (en) 2009-10-26 2016-05-10 Amazon Technologies, Inc. Provisioning and managing replicated data instances
US8335765B2 (en) 2009-10-26 2012-12-18 Amazon Technologies, Inc. Provisioning and managing replicated data instances
US11477105B2 (en) 2009-10-26 2022-10-18 Amazon Technologies, Inc. Monitoring of replicated data instances
US9806978B2 (en) 2009-10-26 2017-10-31 Amazon Technologies, Inc. Monitoring of replicated data instances
US11321348B2 (en) 2009-10-26 2022-05-03 Amazon Technologies, Inc. Provisioning and managing replicated data instances
US8074107B2 (en) 2009-10-26 2011-12-06 Amazon Technologies, Inc. Failover and recovery for replicated data instances
US8620724B2 (en) * 2010-04-20 2013-12-31 Accenture Global Services Limited Integration framework for enterprise content management systems
US20110258136A1 (en) * 2010-04-20 2011-10-20 Accenture Global Services Gmbh Integration framework for enterprise content management systems
US10379988B2 (en) 2012-12-21 2019-08-13 Commvault Systems, Inc. Systems and methods for performance monitoring
US11301333B2 (en) 2015-06-26 2022-04-12 Commvault Systems, Inc. Incrementally accumulating in-process performance data and hierarchical reporting thereof for a data stream in a secondary copy operation
US10275320B2 (en) 2015-06-26 2019-04-30 Commvault Systems, Inc. Incrementally accumulating in-process performance data and hierarchical reporting thereof for a data stream in a secondary copy operation
US10176036B2 (en) 2015-10-29 2019-01-08 Commvault Systems, Inc. Monitoring, diagnosing, and repairing a management database in a data storage management system
US11474896B2 (en) 2015-10-29 2022-10-18 Commvault Systems, Inc. Monitoring, diagnosing, and repairing a management database in a data storage management system
US10853162B2 (en) 2015-10-29 2020-12-01 Commvault Systems, Inc. Monitoring, diagnosing, and repairing a management database in a data storage management system
US10248494B2 (en) 2015-10-29 2019-04-02 Commvault Systems, Inc. Monitoring, diagnosing, and repairing a management database in a data storage management system
US11200110B2 (en) 2018-01-11 2021-12-14 Commvault Systems, Inc. Remedial action based on maintaining process awareness in data storage management
US11815993B2 (en) 2018-01-11 2023-11-14 Commvault Systems, Inc. Remedial action based on maintaining process awareness in data storage management
US10831591B2 (en) 2018-01-11 2020-11-10 Commvault Systems, Inc. Remedial action based on maintaining process awareness in data storage management
US11449253B2 (en) 2018-12-14 2022-09-20 Commvault Systems, Inc. Disk usage growth prediction system

Also Published As

Publication number Publication date
US20030187945A1 (en) 2003-10-02
US6947981B2 (en) 2005-09-20

Similar Documents

Publication Publication Date Title
US6947981B2 (en) Flexible data replication mechanism
US6934826B2 (en) System and method for dynamically allocating memory and managing memory allocated to logging in a storage area network
US6880052B2 (en) Storage area network, data replication and storage controller, and method for replicating data using virtualized volumes
US7137032B2 (en) System and method for ensuring merge completion in a storage area network
US6928513B2 (en) System and method for managing data logging memory in a storage area network
US7542987B2 (en) Automatic site failover
US6996672B2 (en) System and method for active-active data replication
US7415506B2 (en) Storage virtualization and storage management to provide higher level storage services
US7149769B2 (en) System and method for multi-destination merge in a storage area network
US7340640B1 (en) System and method for recoverable mirroring in a storage environment employing asymmetric distributed block virtualization
US6571354B1 (en) Method and apparatus for storage unit replacement according to array priority
US8166128B1 (en) Systems and methods for dynamically updating a virtual volume in a storage virtualization environment
US7383381B1 (en) Systems and methods for configuring a storage virtualization environment
US7290168B1 (en) Systems and methods for providing a multi-path network switch system
US7236987B1 (en) Systems and methods for providing a storage virtualization environment
US7337351B2 (en) Disk mirror architecture for database appliance with locally balanced regeneration
US9009427B2 (en) Mirroring mechanisms for storage area networks and network based virtualization
US7281158B2 (en) Method and apparatus for the takeover of primary volume in multiple volume mirroring
US20070094466A1 (en) Techniques for improving mirroring operations implemented in storage area networks and network based virtualization
US20070094464A1 (en) Mirror consistency checking techniques for storage area networks and network based virtualization
US20060218360A1 (en) Method, apparatus and program storage device for providing an optimized read methodology for synchronously mirrored virtual disk pairs
US7702757B2 (en) Method, apparatus and program storage device for providing control to a networked storage architecture
US20090259816A1 (en) Techniques for Improving Mirroring Operations Implemented In Storage Area Networks and Network Based Virtualization
US20080270727A1 (en) Data transfer in cluster storage systems
US7542986B2 (en) System and method for maintaining order for a replicated multi-unit I/O stream

Legal Events

Date Code Title Description
STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION