|Publication number||US7698318 B2|
|Application number||US 11/351,547|
|Publication date||Apr 13, 2010|
|Filing date||Feb 10, 2006|
|Priority date||Feb 10, 2006|
|Also published as||CN101385005A, CN101385005B, EP1982263A1, EP1982263A4, EP1982263B1, US20070192386, WO2007094904A1|
|Publication number||11351547, 351547, US 7698318 B2, US 7698318B2, US-B2-7698318, US7698318 B2, US7698318B2|
|Inventors||Robert M. Fries, Vinay S. Badami, Michael L. Michael, Shiraz M. Somji|
|Original Assignee||Microsoft Corporation|
|Export Citation||BiBTeX, EndNote, RefMan|
|Patent Citations (18), Non-Patent Citations (2), Referenced by (13), Classifications (10), Legal Events (4)|
|External Links: USPTO, USPTO Assignment, Espacenet|
Background and Relevant Art
As computerized systems have increased in popularity, so have the needs to store and backup electronic files and other communications created by the users and applications associated therewith. In general, computer systems and related devices create files for a variety of reasons, such as in the general case of creating a word processing document in a work setting, as well as creating a file used for more sophisticated database purposes. In addition, many of these documents can include valuable work product, or sensitive information that should be protected.
One will appreciate, therefore, that there are a variety of reasons why an organization will want to backup electronic files on a regular basis, and thereby create a reliable restoration of an originally created file when needed. Generally, some of the challenges facing organizations implementing one or more such backup solutions relate to choices in a particular replication mechanism. That is, there are many ways (i.e., replication mechanisms) to copy data to be protected from a production server volume to a backup storage volume at a backup server, which is where the protected data would reside for recovery purposes. One can appreciate that each replication mechanism carries with it certain advantages and disadvantages.
For example, one conventional replication mechanism involves the production server logging the names of files that have changed on a volume to be protected, and then sending the entire, updated files to a backup volume at the backup server that corresponds to the volume to be protected at the production server. Another, similar mechanism for doing this is for the production server to not only log the name(s) of file(s) that have changed, but also compare the file(s) that have changed at the production server with any corresponding backup copy(ies) of the file(s) at the backup server, and then send to the backup server only the differential, changed bytes.
In particular, the latter mechanism can allow for faster monitoring in part since it may be done without use of a file system filter to monitor changes. Unfortunately, this replication mechanism may involve more resource overhead when comparing a prior copy of the file with an updated version. As such, both of these types of replication mechanism tend to be more effective with smaller files, or with large files that only have a set of the same bytes in a block of bytes that change frequently. Conversely, these replication mechanisms can be very inefficient for very large files, such as database files, particularly files that have sets of several bytes or byte blocks that change with relatively low frequency.
Another conventional replication mechanism involves identifying changes to files, rather than identifying only files that have changed. This mechanism of identifying changes to files typically relies on identifying files (e.g., names, types or locations) that are intended for replication, and identifying only the bytes that have changed in the file between administrator-defined time intervals in between replications. Thus, a backup agent (e.g., a “clone agent” in combination with a “file system filter” at the production server) logs only those changed bytes in the file, and ultimately communicates those changed bytes to the backup storage volume (i.e., “replica volume” on the storage medium). Unfortunately, this replication mechanism still tends to be more cost-effective from a resource expenditure standpoint for very large files or files that change infrequently between replication intervals, but less cost-effective for files that tend to change frequently or are entirely overwritten with each update.
Still another type of replication mechanism, which could be considered a hybrid in some respects of both of the above-discussed replication mechanisms, involves identifying files in terms of “byte blocks.” Generally, “byte blocks” comprise fixed size contiguous blocks of bytes, of which there can be many in any given file. For example, a production server (or “file server”) can identify files as sets of multiple blocks, where each block contains a plurality of bytes. If any of the bytes change within a given block (i.e., are updated, written to, etc.), the replication mechanism might flag the changed block, and send the entire block to the replica volume at an appropriate time. As such, the replica agent can spend only those resources that may be necessary to identify a changed block of bytes, rather than each changed byte in the file. This can allow a given server to avoid incurring additional overhead even though multiple changes may be made to the same byte block. Nevertheless, while this can provide the replication agent with some resource-expenditure advantages over the aforementioned mechanisms, this mechanism may still be better suited for larger files, such as database files, or files whose byte blocks are changed more than once within the same replication cycle.
Accordingly, an organization that is determining to use a particular replication mechanism for its backup service may need to weigh several considerations. Complicating this is the notion that, even though an organization may make a determination on its present file generation/change needs, such a consideration may nevertheless be inadequate in the future. For example, the organization's determination of a particular replication mechanism will typically be applied to all files to be protected, without regard to indicia that may make the determination more applicable for some files than for others, such as file type, size, location, or the like. Thus, the determination may be based on what the organization feels is best with its current environment, such as the set of most common file types, and/or commonly used applications.
Of course, if the predominant file type(s) and/or application types change(s) at a later point, then it is possible that the initially chosen replication mechanism may need to be replaced. This possibility can make it particularly difficult for the organization, both at the outset when trying to project what replication mechanism will be preferred, as well as at a later point from a resource expenditure perspective if or when needing to change. For example, the organization could insist that the bulk of applications used in the organization use a certain file type and/or application type that is suited to the chosen replication mechanisms, or alternatively commit itself to changing its replication mechanism periodically. Both of these scenarios, of course, can lead to significant cost and resource expenditure problems for the organization.
Implementations of the present invention solve one or more problems in the art with systems, methods, and computer program products configured to provide efficient determinations of appropriate replication mechanism for files in a production server. In particular, implementations of the present invention allow a determination to be made differently per file, per location, per file type, or per some other criterion, such that several different files on a production server could be backed up using different replication mechanisms. Furthermore, implementations of the present invention allow for such determinations to fluctuate automatically over time, to thereby ensure that the production server continues to use the most efficient replication mechanism for each file.
For example, a method in accordance with at least one implementation of the present invention from the perspective of a data protection manager server (i.e., backup server) for automatically determining an appropriate replication mechanism can involve identifying a plurality of files of a production server to be protected. The method can further involve identifying first replication information for a first set of one or more files in the plurality of files, as well as identifying second replication information for a second set of one or more files in the plurality of files. In addition, the method can involve assigning the first replication mechanism to the first set of files based on the first replication information.
Furthermore, the method can involve assigning the second replication mechanism to the second set of files based on the second replication information. As such, the first set of files and the second set of files are assigned in a way that they are to be replicated using different replication mechanisms. Upon making these assignments, the method can also involve sending the first replication mechanism assignment and the second replication assignment to a production server.
In addition, a method in accordance with an implementation of the present invention from the perspective of a production server (i.e., file server) for backing up file changes to the replica volume can involve identifying a plurality of files to be protected in a file system at the production server. The method can further involve receiving an indication that a first set of one or more files in the plurality of files is assigned to be replicated using a first replication mechanism. In addition, the method can involve receiving an indication that a second set of one or more files in the plurality of files is assigned to be replicated using a second replication mechanism. As such, the first set of one or more files and the second set of one or more files are assigned to be replicated using different replication mechanisms. In addition, the method can involve logging byte data of changes to files in the first set of files, as well as logging names of files that have changed in the second set of files.
Additional features and advantages of exemplary implementations of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of such exemplary implementations. The features and advantages of such implementations may be realized and obtained by means of the instruments and combinations particularly pointed out in the appended claims. These and other features will become more fully apparent from the following description and appended claims, or may be learned by the practice of such exemplary implementations as set forth hereinafter.
In order to describe the manner in which the above-recited and other advantages and features of the invention can be obtained, a more particular description of the invention briefly described above will be rendered by reference to specific embodiments thereof which are illustrated in the appended drawings. Understanding that these drawings depict only typical embodiments of the invention and are not therefore to be considered to be limiting of its scope, the invention will be described and explained with additional specificity and detail through the use of the accompanying drawings in which:
The present invention extends to systems, methods, and computer program products configured to provide efficient determinations of appropriate replication mechanism for files in a production server. In particular, implementations of the present invention allow a determination to be made differently per file, per location, per file type, or per some other criterion, such that several different files on a production server could be backed up using different replication mechanisms. Furthermore, implementations of the present invention allow for such determinations to fluctuate automatically over time, to thereby ensure that the production server continues to use the most efficient replication mechanism for each file.
As will be appreciated more fully from the following specification and claims, data to be protected at a production server can be replicated on any of a plurality of different bases. In some cases, the administrator can input how a given set of files are to be replicated, while in other cases, the determinations can be made automatically (by a DPM server, or by a production server) based on some file use characteristics. For example, one result of backup administrator input, or of some automatic determination, might be to indicate that all database files (e.g., with a “.db” file extension) are to be replicated using an identification of changed bytes. Another result of a determination might be to indicate that all other files (e.g., those with a “.doc” extension, or those in a particular folder location) are backed up by replicating updated copies of the entire file. Still further, other sets of files can be set to be replicated based on determinations of their file size, location in the file system, and frequency of updates.
If the backup server made the determinations, then the backup server can then transmit this information to the production server. When the backup server requests the updates of each file, the production server can either send over copies of the changed file bytes, entire copies of the changed file itself, or even changed blocks of a file, as appropriate. As such, one will appreciate from the description herein that an organization can gain efficiency by automating selection and implementation of a wide variety of replication mechanisms with a variety of different files at a production server.
In any event, of the illustrated replication mechanisms, replication mechanism 140 relates to “changes to file,” which in this case means identification and replication of the particular bytes in a file that have changed. Replica agent 130 might select replication mechanism 140 for particularly large files, where it is more efficient to send only the changed raw bytes for the file over a network connection. Another of the replication mechanisms includes mechanism 145, which relates to “files that have changed.” Generally, replication mechanism 145 refers to entire files (typically much smaller files, such as word processing files) that can be copied and sent in whole part to storage medium 160 when production server 105 determines that any portion of the file has changed. This can be done any number of ways, including sending an entire, updated file to storage medium 160, or by comparing the updated file to a backup copy of the file, and sending over only the changed bytes to storage medium 160. In both cases, specific bytes of the updated file are not logged in a log file when they are updated.
Still another of the illustrated replication mechanisms includes mechanism 150, which relates to “changes to blocks.” Generally, each file can be thought of as a set of byte blocks. When any byte in a particular block has been updated, the production server can log the file name, as well as the byte block (i.e., typically this has a fixed size and consists of a collection of from 4096 to 16384 bytes) that has changed, and ultimately send that byte block to DPM server 110 when appropriate. Thus, replication mechanism 150 can be thought of as potentially replicating more data than otherwise might be sent with replication mechanism 140 (i.e., “changes to file”) when the changes to a file are relatively infrequent on the same byte block. At the same time, replication mechanism 150 might be thought of as potentially less data than might otherwise be sent with replication mechanism 145 (i.e., “files that have changed”), unless sending only the changed bytes as described previously. One will appreciate therefore, that each described replication mechanism 140, 145, 150, can provide its own unique advantages, depending on file usage or system needs, and the way in which the replication mechanism is implemented.
In any event, replica agent 130 can associate various files at production server 105 with a particular replication mechanism, based on any number of automatic (static and/or dynamic) factors. For example,
In particular, one will appreciate that file system filter 123 can be configured any number of ways to implement an assigned replication mechanism. In one particular implementation, for example, file system filter 123 continues to log (i.e., “capture”) the data for each write to a special log file, such as files assigned to replication mechanism 140. File system filter 123 can then mark certain portions of other files, such as files assigned to replication mechanism 145 or 150, as dirty when updated. File system filter 123 can do this such as by marking that the particular file has changed, or that certain blocks of the file has changed. In addition, and rather than logging the actual changed data when using replication mechanisms 145 or 150, file system filter 123 can simply log the file names of the changed files, as well as the byte block addresses. When DPM server 110 requests updates from production server 110, clone agent 127 can send over the byte data in the log file, or send a copy of the file (or the changed file block(s)) identified by name in the log file.
Accordingly, the illustrated implementation shows that file system filter 123 adds byte data to one log file (i.e., 170), but adds only the file names or block addresses in a different log file (i.e., 175). One will appreciate, however, that it is not necessary that various data changes be logged in separate files, or that the different log files be constructed using differing data change identification mechanisms. For example, file system filter 123 can log the changed bytes—as well as the file names and block addresses of changed files—in the same log file (e.g., 170 or 175), in accordance with implementations of the present invention. Similarly, file system filter 123 could also log byte addresses and files names in lieu of actual changed byte data; while, at the same time, file system filter 123 could log data for an entire file or for an entire block in a given log file (e.g., 170 and/or 175).
Nevertheless, and with respect to the illustrated implementation, clone agent 127 can simply forward log 170 containing the byte data to replica agent 130 when appropriate. With respect to log 175, clone agent 127 can first identify within log 175 whether a file or file block has changed. Upon so identifying, clone agent 127 can then copy the identified file or changed file blocks from their respective file system locations, and forward those changed files or file blocks to replica agent 130. In turn, replica agent 130 can then pass the data received from clone agent 127 to storage medium 160. As such, replica agent 130, clone agent 127, and file system filter 123 concertedly implement a different replication mechanism for each of files (or file sets) 115, 120, and 125.
As previously mentioned, these different assignments of replication mechanisms to one or more files or file sets can be done automatically. For example, file system filter 123 may, at some point, identify and pass along replication information to clone agent 127 and replica agent 130 that indicates that file 115 is shrinking to a much smaller size. Similarly, file system filter 123 might identify and pass along replication information to replica agent 130, which indicates that file 120 is dramatically increasing in both size and frequency of file updates. Replica agent 130 can be configured, in turn, to evaluate any received replication information, and reevaluate (not shown) a replication mechanism assignment for a given file or file set. Replica agent 130 can also be configured to prompt a backup administrator to provide new input regarding prior replication mechanism assignments based on new information.
Accordingly, replica agent 130 can be configured to reassign replication mechanisms for each of the files in production server 105, whether automatically in response to information received from clone agent 127, or in response to new input received periodically from a backup administrator. Furthermore, file and replication mechanism assignments can be easily adjusted as needed to thereby ensure the most efficient use of replication resources in system 100. In particular, implementations of the present invention can enhance efficiency of a backup system at least in part by allowing backups to occur with reduced consumption of network bandwidth, reduced amounts of local storage needed, and reduced local replication CPU overhead.
In such a case, DPM server 110 (or the like) might simply be configured to send instructions (e.g., administrator preferences) to the production server 105 regarding how to determine what replication mechanism to use (i.e., a default setting, some file behavior patterns, or the like). The production server 105 could then be configured to automatically determine and adjust what replication assignments are used for each given file (or file set) on its own, as opposed to the more passive role generally described herein. These general illustrations and descriptions, therefore, present only some of several possible implementations in accordance with the present invention for automatically determining one or more replication mechanisms for a given file (or file set) from multiple possible replication mechanisms.
In addition to the foregoing overview schematic diagrams, implementations of the present invention can also be described in terms of methods comprising a sequence of one or more acts for accomplishing a particular result. In particular,
As a preliminary matter,
In any event,
Accordingly, the schematics, components, and methods illustrated or described herein provide a number of mechanisms for ensuring that a DPM server (e.g., 110) can implement a variety of replication mechanisms in a way that is most efficient, and appropriately tailored for file usage at a production server. Thus, an organization can avoid committing to a particular replication mechanism at any given time. Furthermore, backup administrators can avoid the loss of resources that might otherwise be needed at a future point when updating or significantly changing from one replication mechanism scheme to another replication mechanism scheme.
One will appreciate, further, that the replication mechanisms described herein are simply exemplary types of replication mechanisms that can be considered by determination module 135 in accordance with implementations of the present invention. In particular, an organization may have many more replication mechanisms, and/or ways of replicating data updates, that it might desire to use at a production server. Implementations of the present invention are not limited to greater or fewer numbers than those replication mechanisms described herein, or to the particular time of replication mechanisms described herein. Rather, at least one advantage of implementations of the present invention is the ability to continually choose a most appropriate replication mechanism from what replication mechanisms are available in consideration of present or projected characteristics of files in a file system.
Embodiments within the scope of the present invention also include computer-readable media for carrying or having computer-executable instructions or data structures stored thereon. Such computer-readable media can be any available media that can be accessed by a general purpose or special purpose computer. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to carry or store desired program code means in the form of computer-executable instructions or data structures and which can be accessed by a general purpose or special purpose computer. When information is transferred or provided over a network or another communications connection (e.g. via a hardwired connection) to a computer, the computer properly views the connection as a computer-readable medium. Thus, any such connection is properly termed a computer-readable medium. Combinations of the above should also be included within the scope of computer-readable media.
Computer-executable instructions comprise, for example, instructions and data which cause a general purpose computer, special purpose computer, or special purpose processing device to perform a certain function or group of functions. Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.
The present invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described embodiments are to be considered in all respects only as illustrative and not restrictive. The scope of the invention is, therefore, indicated by the appended claims rather than by the foregoing description. All changes that come within the meaning and range of equivalency of the claims are to be embraced within their scope.
|Cited Patent||Filing date||Publication date||Applicant||Title|
|US6625623||Dec 16, 1999||Sep 23, 2003||Livevault Corporation||Systems and methods for backing up data files|
|US6704755||May 21, 2002||Mar 9, 2004||Livevault Corporation||Systems and methods for backing up data files|
|US6804755||Jun 19, 2001||Oct 12, 2004||Storage Technology Corporation||Apparatus and method for performing an instant copy of data based on a dynamically changeable virtual mapping scheme|
|US6857053 *||Apr 10, 2002||Feb 15, 2005||International Business Machines Corporation||Method, system, and program for backing up objects by creating groups of objects|
|US6898600 *||May 16, 2002||May 24, 2005||International Business Machines Corporation||Method, system, and program for managing database operations|
|US6931422 *||Nov 12, 1999||Aug 16, 2005||International Business Machines Corporation||Enhanced backup and recovery methodology|
|US7007144 *||Dec 24, 2003||Feb 28, 2006||Hitachi, Ltd.||Method, apparatus, and computer readable medium for managing back-up|
|US7159081 *||Aug 28, 2003||Jan 2, 2007||Hitachi, Ltd.||Automatic scenario management for a policy-based storage system|
|US20020065835 *||Mar 29, 2001||May 30, 2002||Naoya Fujisaki||File system assigning a specific attribute to a file, a file management method assigning a specific attribute to a file, and a storage medium on which is recorded a program for managing files|
|US20020147734 *||Apr 6, 2001||Oct 10, 2002||Shoup Randall Scott||Archiving method and system|
|US20020156542 *||Feb 23, 2001||Oct 24, 2002||Nandi Hill K.||Methods, devices and systems for monitoring, controlling and optimizing processes|
|US20040107199 *||Jul 22, 2003||Jun 3, 2004||Mdt Inc.||Computer application backup method and system|
|US20050033757 *||Jun 24, 2004||Feb 10, 2005||Arkivio, Inc.||Techniques for performing policy automated operations|
|US20050160118 *||Feb 14, 2005||Jul 21, 2005||Microsoft Corporation||System and method for backing up and restoring data|
|US20050192985 *||Apr 15, 2005||Sep 1, 2005||Fujitsu Limited||Apparatus and method for classifying files, and computer product|
|US20050203908 *||Mar 12, 2004||Sep 15, 2005||Sahn Lam||Managing data replication policies|
|US20070022087 *||Jul 25, 2006||Jan 25, 2007||Parascale, Inc.||Scalable clustered storage system|
|KR20040108766A||Title not available|
|1||*||"Replication techniques for speeding up parallel applications on distributed systems", by Bal et al., Concurrency Practice and Experience. vol. 4, No. 5, pp. 337-355. 1992.|
|2||*||Sivasubramanian, S., Pierre, G., and Van Steen, M., 2003. A case for dynamic selection of replication and caching strategies. In Proc. 8th Web Caching Workshop (Hawthorne, NY).|
|Citing Patent||Filing date||Publication date||Applicant||Title|
|US7831859 *||Jun 17, 2008||Nov 9, 2010||The Board Of Regents, University Of Texas System||Method for providing fault tolerance to multiple servers|
|US7882064 *||Jul 6, 2006||Feb 1, 2011||Emc Corporation||File system replication|
|US8041678 *||Jun 20, 2008||Oct 18, 2011||Microsoft Corporation||Integrated data availability and historical data protection|
|US8315986||Dec 24, 2007||Nov 20, 2012||Emc Corporation||Restore optimization|
|US8838630 *||May 28, 2010||Sep 16, 2014||Citrix Systems, Inc.||Method and systems for efficient delivery of previously stored content|
|US8943304||Aug 3, 2006||Jan 27, 2015||Citrix Systems, Inc.||Systems and methods for using an HTTP-aware client agent|
|US9407608||Jul 31, 2014||Aug 2, 2016||Citrix Systems, Inc.||Systems and methods for enhanced client side policy|
|US9563655 *||Mar 8, 2013||Feb 7, 2017||Oracle International Corporation||Zero and near-zero data loss database backup and recovery|
|US20080010322 *||Jul 6, 2006||Jan 10, 2008||Data Domain, Inc.||File system replication|
|US20080320259 *||Jun 17, 2008||Dec 25, 2008||Garg Vijay K||Method for Providing Fault Tolerance to Multiple Servers|
|US20090319583 *||Jun 20, 2008||Dec 24, 2009||Microsoft Corporation||Integrated data availability and historical data protection|
|US20100235374 *||May 28, 2010||Sep 16, 2010||Henk Bots||Method and Systems for Efficient Delivery of Previously Stored Content|
|US20140258241 *||Mar 8, 2013||Sep 11, 2014||Oracle International Corporation||Zero and near-zero data loss database backup and recovery|
|U.S. Classification||707/610, 707/637, 707/633, 707/638|
|International Classification||G06F7/00, G06F17/00|
|Cooperative Classification||G06F11/1464, G06F11/1451|
|European Classification||G06F11/14A, G06F11/14A10D2|
|Feb 27, 2006||AS||Assignment|
Owner name: MICROSOFT CORPORATION, WASHINGTON
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:FRIES, ROBERT M.;BADAMI, VINAY S.;MICHAEL, MICHAEL L.;AND OTHERS;REEL/FRAME:017221/0525;SIGNING DATES FROM 20060207 TO 20060210
Owner name: MICROSOFT CORPORATION,WASHINGTON
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:FRIES, ROBERT M.;BADAMI, VINAY S.;MICHAEL, MICHAEL L.;AND OTHERS;SIGNING DATES FROM 20060207 TO 20060210;REEL/FRAME:017221/0525
|Jan 25, 2011||CC||Certificate of correction|
|Sep 25, 2013||FPAY||Fee payment|
Year of fee payment: 4
|Dec 9, 2014||AS||Assignment|
Owner name: MICROSOFT TECHNOLOGY LICENSING, LLC, WASHINGTON
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:MICROSOFT CORPORATION;REEL/FRAME:034543/0001
Effective date: 20141014