|Publication number||USRE39458 E1|
|Application number||US 09/512,967|
|Publication date||Jan 2, 2007|
|Filing date||Feb 24, 2000|
|Priority date||Jan 14, 1994|
|Also published as||CA2140124A1, CA2140124C, DE69534292D1, DE69534292T2, EP0665670A2, EP0665670A3, EP0665670B1, EP0665670B8, US5446888, US5721907|
|Publication number||09512967, 512967, US RE39458 E1, US RE39458E1, US-E1-RE39458, USRE39458 E1, USRE39458E1|
|Inventors||Charles F. Pyne|
|Original Assignee||Laplink Software, Inc.|
|Export Citation||BiBTeX, EndNote, RefMan|
|Patent Citations (20), Referenced by (8), Classifications (15), Legal Events (6)|
|External Links: USPTO, USPTO Assignment, Espacenet|
This is a continuation of application Ser. No. 08/182,969, filed Jan. 14, 1994 now U.S. Pat. No. 5,446,888.
The present invention relates to computer communications in general and, in particular, to a method and apparatus for decreasing the time required to update files located at a remote computer.
In computer communications technology, the rate of data communication between a computer and other peripheral devices is very important. The ability to quickly and accurately transfer data between two personal computers is of special interest in light of the increased use of portable computers. Often, data entered into a portable computer is ultimately transferred to a user's home or office personal computer. Computer specialists are continually searching for communication protocols that decrease the time required to transfer data without compromising the reliability of the data being transmitted.
A conventional method for conveying data between computers, especially personal computers, involves the interconnection of a data bus in a sending computer with a data bus in a receiving computer. This may be done by coupling the serial, parallel, or similar communications ports of each computer through an interface link, such as a cable or across a data path using modems. In serial communication, data is transferred one bit at a time. Serial communications work well for transferring data over long distances, and particularly with modems that couple two computers using a telephone line. However, the time required to transfer data using serial communications can be significant, especially for larger files. When communicating between two devices that are relatively close, parallel communications are often used. Parallel communication is the simultaneous transfer of a number of bits of data in parallel, e.g., 8-bit, using a multi-bit data path.
Computer software companies are continually investigating more efficient methods of transferring data to reduce data transmission times. Two prevalent areas of concentration have been on increasing data transfer rates and on incorporating forms of data compression to reduce the amount of data being sent. Advances in data transfer rates have been accomplished by increasing the speed at which modems communicate in serial communication and by increasing the number of bits that can be transferred simultaneously in parallel communication. An example technology that incorporates the latter technique is described in U.S. Pat. No. 5,261,060, titled “Eight-bit Parallel Communications Method and Apparatus,” and assigned to the assignee of the present invention. U.S. Pat. No. 5,261,060 is hereby incorporated by reference. Data compression schemes reduce the size of a file to be transmitted by various means of compacting information. For example, one common compression technique, called key-word encoding, replaces words that occur frequently, e.g., the, with a 2-byte token representation of each word. After the compressed data is received by a remote computer, the data is decompressed to create a representation of the original contents of the file.
A more recent approach to decreasing the time required to transfer a file has recognized that a receiving computer will often have a file, i.e., a reference file, that is similar or perhaps even identical to a source file to be transmitted. For example, the source file may simply include text from the reference file with only a few words or sentences changed. Rather than sending an original or compressed representation of the entire source file, file transfer methods utilizing this approach identify the differences between the two files, and then transfer only the differences to thee receiving computer. Upon receipt, the difference information is used to update the reference file at the receiving computer, thereby reproducing a precise copy of the source file. The present invention is directed toward an improved method of identifying and transferring revisions between a source file and a reference file to create an accurate copy of the source file at a remote computer.
The invention is a file transfer method that identifies and isolates the differences between a source file located at a sending computer and a reference file, located at a receiving computer, that may have data similar to the data comprising the source file. The computers are connected through a computer data interface. The method includes the steps of: (a) dividing the reference file into a plurality of data blocks and associating each data block with a key value representative of the data in each block; and (b) identifying blocks of data at the source file and using the key values to compare blocks of data from the reference file to blocks of data from the source file and, in instances where a match is found between a block of data from each file, sending an indication of the match to the receiving computer so that the block of data indicated by the match need not be transmitted to the receiving computer.
In accordance with other aspects of the invention, the step of identifying blocks of data at the source file includes the step of computing a source key for each block of data which is then compared to the key values from the reference file.
In accordance with still further aspects of the invention, an initial block of data is identified from the source file and a source key is computed from the initial block. If a match for the initial block is not found, the method includes the steps of: (i) transmitting a byte of data from the initial block to the receiving computer; and (ii) identifying a subsequent block of data from the source file comprising the initial block of data, less the transmitted byte, and a byte of data from the source file.
In accordance with other aspects of the invention, the method includes the step of transmitting the key values associated with data blocks in the reference file to the sending computer. Further, the key value for a block of data is computed by multiplying the bytes in the block by one or more multipliers, the value of the multiplier being dependent upon the position of a given byte in the block, and summing the results of the multiplication operations.
The foregoing aspects and many of the attendant advantages of this invention will become more readily appreciated as the same becomes better understood by reference to the following detailed description, when taken in conjunction with the accompanying drawings, wherein:
Remote file transfer applications often involve a situation where a receiving computer already contains a file that is similar, or perhaps even identical to, a file to be transmitted. For example, the file to be transmitted may be a revision of a text file with only a few words or sentences changed. The invention is a file transfer method that identifies and isolates the differences between the two files, and transmits only those differences to the receiving computer. For similar files, the file transfer method can result in compression ratios far in excess of those achieved by traditional data compression methods.
The sending computer 20 generally comprises a processing unit 26, a memory 28, and a number of communications ports 30. The memory, including random access memory (RAM), read only memory (ROM), and external systems memory, is connected to the processing unit 26 by a data/address bus 32. The communications ports are connected to the processing unit by a data bus 34. The communications ports 30 include parallel and serial ports, as well as other input/output technologies including PCMCIA card technology, that allow data to be sent and received by the sending computer. The receiving computer 22 is similar to the sending computer, and includes a processing unit 36, memory 38, communications ports 40, data/address lines 42, and a data bus 44. Although for ease of description one computer is called the sending computer and the other is called the receiving computer, the computers are generally interchangeable.
In order to accomplish data transfer, the sending and the receiving computers include computer program controls that, for example, are stored in RAM and executed by the processing units of each computer. In one embodiment of the invention, the sending and receiving computer controls are combined into a single file transfer program 45 that is resident at each computer. In this manner, each computer can operate as a sending or receiving computer. Because of the requirements of handshaking, copies of the file transfer program 45 located at each computer are preferably executed simultaneously. This allows for full-duplex transmission, i.e., simultaneous communications in each direction. The invention may also be utilized in half-duplex communications, although not as efficiently.
For clarity in this discussion, throughout the Detailed Description it is assumed that a source file 46 located at the sending computer is to be sent to the receiving computer 22. Further, it is assumed that the receiving computer includes a reference file 48 that includes at least some similarities to the source file. Once a user indicates that a source file is to be transferred, a reference file that may have data that is similar to the source file is identified by, for example, having a file name that is the same or similar to the source file. The invention described herein generally assumes that a reference file has been identified. The basic steps implemented by the file transfer program are as follows:
(1) identifying a reference file at the receiving computer that may have data similar to data comprising the source file;
(2) dividing the data comprising the reference file into a plurality of data blocks having n-bytes per block and associating each data block with a key value;
(3) transmitting the key values from the receiving computer to the sending computer;
(4) identifying a current n-byte block of data from the source file and computing a value for a source key associated with the current block of data;
(5) comparing the value of the source key with each of the key values from the reference file and, if a match is found, (i) transmitting an indication of the match to the receiving computer, and (ii) repeating step (4); and
(6) if a match was not found, transferring to the receiving computer a byte of data from the current block of data, adding an additional byte of data from the source file to the current block of data, re-computing the value of the source key, and repeating step (5).
Generally, the loops created by steps (5) and (6) repeat until all of the data in the source file has been considered. At the receiving computer, a destination file is created from the match indications and the byte transmissions. The destination file will be a duplicate of the source file upon completion of transmission.
Once the keys for each block of data in the reference file have been computed, the keys are sent as an array to the sending computer for comparison to the source file.
In step (4), the value of KEY1 is compared to each of the keys in the reference file to determine whether a match has been found, thereby indicating that the current block of data is identical to a block of data in the reference file. If a match is found, an indication of such is sent to the receiving computer. Assuming a match has not been found, according to step (5) the first byte in the current block (byte zero) is sent to the receiving computer. A subsequent “current” block of data is then evaluated by subtracting the first byte of data (byte zero) from the current block, adding the next sequential byte of data (byte n) to the current block, and recomputing the key value for the subsequent current block. The key value for this block of data is labeled KEY2. Thus, KEY2 will comprise the values of bytes 1 through n. The value of KEY2 is then compared to each of the keys in the key array for the reference file.
Assuming a match is not made, the first byte in the current block (byte 1) is sent to the receiving computer. A third key KEY3 representing the current block of data is then computed by sliding the current block of data one byte to the right, such that KEY3 comprises the values of bytes 2 through (n+1). This will continue until either a match is found between a key value computed from a data block in the source file and a key in the key array for the reference file, or all of the data in the source file has been transmitted. Assuming a match is found, an indication of such is sent to the receiving computer, and a subsequent current block of data is computed from the source file.
It is noted that in the case where a match is not found, the additional time required to transfer a file, in comparison to traditional methods, is negligible despite the sliding window and key computations. This is, in part, due to the fact that the processor can make computations much faster than data can be sent. Further, in a preferred embodiment, the receiving computer is configured to expect that bytes being received are data bytes and are not indicative of a match between two data blocks. In the latter case, an additional “match-indicator” byte is sent ahead of the byte(s) indicating that a match has occurred. Thus, the number of bytes being sent in the case of no matches will generally be approximately the same as if the data were simply transmitted without any opportunity for match checks in accordance with the invention.
The foregoing is an overview of an exemplary embodiment of the file transfer program 45. Exemplary routines for implementing the file transfer program is software are set forth in
A test is made at block 130 to determine whether the end of the reference file has been reached. If the end of the reference file has not been reached, a test is made at block 132 to determine whether a full block of data has been considered, i.e., whether n is equal to the block size. If n is not equal to the block size, the subroutine loops back to block 124. If n is equal to the block size, or if it was determined at block 130 that the end of the file was reached, the subroutine terminates, and control returns to the routine of FIG. 4.
If a match was found at block 160, a message is sent at block 162 to the receiving computer to emit the matching block to the destination file. The routine then loops to block 154. If a match was not found, at block 164 a first byte of data in the current block is sent to the receiving computer. At block 166, the byte of data that was sent to the receiving computer is removed from the current block of data. A test is then made at block 168 to determine whether there is any data remaining in the source file that has not been considered. If there is data remaining in the source file, a new byte of data is read from the source file at block 170 and added to the current block of data at block 172. The routine then loops to block 156 where the key for the current block of data is computed.
Those skilled in the art will appreciate that, in computing the current key in block 156 after looping from block 172, it is more efficient to obtain the value of the key for the current block by subtracting from the previously-computed current key the value of the byte that was removed from the current block (in block 166) and then adding the value of the byte that was added to the current block (in block 172), rather than performing the key calculation by adding every character in the current block.
If all of the data remaining in the source file has been considered, or if there were less than n-bytes of data remaining in the source file as determined in the test at block 152, the data remaining in the source file is sent to the receiving computer at block 174. At the receiving computer, the transmitted data is added to the destination file and the file transfer is complete. In an alternative embodiment, instead of simply sending the data remaining as indicated in block 174, a test is made to determine if the key value of the data remaining matches the key value of the last block of data in the reference file. If a match is present, an indication of the match is sent to the receiving computer, and the data itself need not be transmitted. Otherwise, the actual data is transmitted. This will result in a further optimization of the transfer in situations where the end of the reference file contains the same data as the end of the source file.
Once all of the data in the source file has been added to the destination file, i.e., through blocks 162, 164 and 174, the destination file will in most circumstances be an exact copy of the source file. However, it is preferable that a check be made to ensure that the destination file is indeed a precise duplicate of the source file. In block 176, the integrity of the destination file is checked using means known to those skilled in the art. One method of checking the file integrity is a cyclic redundancy-check (CRC), such as that set forth in M. Nelson, The Data Compression Book, 446-448 (M&T Books 1991), which is hereby incorporated by reference. If the integrity of the destination file was compromised, the data from the source file is retransmitted to the destination file using conventional transmission methods. This is indicated at block 178. If the integrity of the destination file tested positive, or upon transmitting the source file, the routine terminates.
One circumstance where the destination file may not be an accurate copy of the source file is where two or more different blocks of data yield the same key value. If it is assumed that each block of data is 256 bytes, under the key computation method described in
Another desirable feature of an advantageous key computation method is if the current key values for blocks of source data that are derived on a sliding window basis can be quickly established. One way of accomplishing this is to have a key computation method that allows the current key to be updated by subtracting the key value associated with the byte of data to be subtracted from the current block of data (block 166) and adding the key value associated with the byte of data to be added to the current block (block 172). While the key computation method described in
With reference to
C1(n)+C2(n−1)+C3(n−2)+. . . Cn−1(2)+Cn (1)
where Ci is the character in the ith position of a current block and n is the number of bytes in each block. The upper 8-bits of the 32-bit key are calculated by performing an exclusive OR operation (XOR) on each of the characters, as shown by the equation:
C1 XOR C2 XOR C3 . . . Cn−1 XOR Cn (2)
Once the lower and upper key values are calculated, the bits are concatenated to form each 32-bit key.
At block 208, the value of the current byte is added to the variable sum. The key.24 variable is then increased by the value of sum at block 210. It will be appreciated that blocks 208 and 210 are alternate methods of computing equation (1) without requiring multiplication operations. At block 212, the variable key.8 is set equal to the previous value of key.8 XOR the current byte.
A test is made at block 214 to determine if the end of the reference file has been reached. If the end of the reference file has not been reached, a test is made at block 216 to determine whether n is equal to the block size. If n is not equal to the block size, the routine loops to block 206. If n is equal to the block size, or if the end of the file has been reached, the variable “key” is set by concatenating the lower 24-bit (key.24) value computed at block 210 with the upper 8-bit (key.8) value computed at block 212. The subroutine then terminates, and the program returns to block 106 of FIG. 4.
If the current block of data was identified on the basis of a previous, unmatched block of data, the first byte from the previous block (termed the removed byte from the operation of block 166) is subtracted from sum at block 224. At block 226, the key.24 is set equal to its previous value less the product of the block size times the value of the removed byte. At block 228, the new byte (added to the current block in block 172) is added to sum. The key.24 variable is then increased by the value of sum at block 230.
At block 232, the variable key.8 is set equal to the previous value of key.8 XOR the removed byte. An exclusive OR operation is then performed between key.8 the new byte. At block 236 the variable “key” is set by concatenating the lower 24-bit (key.24) value computed at block 230 with the upper 8-bit (key.8) value computed at block 234. The subroutine then terminates, and control returns to block 158 of FIG. 6. As can be seen, the subroutine of
As will be appreciated by those skilled in the art, a large number of different key computation methods may be used in accordance with the invention. Thus, the invention is not to be limited by exemplary key calculations illustrated herein. Any key computation that is not unnecessarily time-consuming computationally and that provides a relatively wide range of results may be beneficial. Moreover, the type of key computation used in any particular embodiment may depend upon the block size to achieve optimal results. Another key computation that may be used is to multiply each character in a block by its position in the block, and summing the results of the multiplication operations. Another key computation that may be implemented is the CRC file integrity check discussed above. Although this method is extremely accurate, it may be too slow for many applications.
With reference again to
While the preferred embodiment of the invention has been illustrated and described, it will be appreciated that various changes can be made therein without departing from the spirit and scope of the invention.
|Cited Patent||Filing date||Publication date||Applicant||Title|
|US3311893||Aug 29, 1963||Mar 28, 1967||Sperry Rand Corp||Memory organization wherein only new data bits which are different from the old are recorded|
|US3612660||Sep 8, 1969||Oct 12, 1971||Wendell S Miller||Digital data compaction|
|US4232375||Jun 12, 1978||Nov 4, 1980||Ncr Corporation||Data compression system and apparatus|
|US4412306||May 14, 1981||Oct 25, 1983||Moll Edward W||System for minimizing space requirements for storage and transmission of digital signals|
|US4641274 *||Aug 19, 1985||Feb 3, 1987||International Business Machines Corporation||Method for communicating changes made to text form a text processor to a remote host|
|US4807182 *||Mar 12, 1986||Feb 21, 1989||Advanced Software, Inc.||Apparatus and method for comparing data groups|
|US4891785||Jul 8, 1988||Jan 2, 1990||Donohoo Theodore J||Method for transferring data files between computers in a network response to generalized application program instructions|
|US4914583||Apr 13, 1988||Apr 3, 1990||Motorola, Inc.||Method of indicating processes resident within a cell of a data processing system|
|US5086402||May 30, 1990||Feb 4, 1992||Simware, Inc.||Method for high speed data transfer|
|US5086434||Sep 26, 1986||Feb 4, 1992||Canon Kabushiki Kaisha||Data communication process and apparatus for data transmission in blocks which may include mixed data types|
|US5101348||Jun 23, 1988||Mar 31, 1992||International Business Machines Corporation||Method of reducing the amount of information included in topology database update messages in a data communications network|
|US5155847||Aug 3, 1988||Oct 13, 1992||Minicom Data Corporation||Method and apparatus for updating software at remote locations|
|US5210865 *||Aug 4, 1992||May 11, 1993||Digital Equipment Corporation||Transferring data between storage media while maintaining host processor access for I/O operations|
|US5283646||Apr 9, 1992||Feb 1, 1994||Picturetel Corporation||Quantizer control method and apparatus|
|US5479654 *||Mar 30, 1993||Dec 26, 1995||Squibb Data Systems, Inc.||Apparatus and method for reconstructing a file from a difference signature and an original file|
|US6076084||Apr 30, 1996||Jun 13, 2000||Norton-Lambert Corp.||File transfer method and apparatus utilizing delimiters|
|EP0069212A1||May 5, 1982||Jan 12, 1983||International Business Machines Corporation||Electric circuit connecting devices|
|EP0541281A2||Oct 26, 1992||May 12, 1993||AT&T Corp.||Incremental-computer-file backup using signatures|
|WO1993019420A1||Mar 11, 1993||Sep 30, 1993||Nomadic Systems, Inc.||Remote file access system|
|WO1999002313A1||Jul 2, 1998||Jan 21, 1999||Asea Brown Boveri Ag||Pantograph-driven robot arm and gantry robot|
|Citing Patent||Filing date||Publication date||Applicant||Title|
|US7752267 *||Jul 6, 2010||Minolta Co., Ltd.||Device, method and program product for data transmission management|
|US8055819 *||May 29, 2008||Nov 8, 2011||Nec Corporation||Information processor|
|US8516473||Jan 11, 2005||Aug 20, 2013||S.W. Caldwell & Company Llc||Converting a limited program object to a complete program object|
|US8725685 *||Jan 27, 2005||May 13, 2014||Sony Corporation||Content transfer apparatus and associated methodology of updating contents between devices|
|US20030046352 *||Mar 13, 2002||Mar 6, 2003||Takeo Katsuda||Device, method and program product for data transmission management|
|US20050240938 *||Jan 11, 2005||Oct 27, 2005||Greg Lane||E-commerce system and method relating to program objects|
|US20080222126 *||Jan 27, 2005||Sep 11, 2008||Sony Corporation||Information Processing Device, Information Processing Method, and Computer Program|
|US20100169520 *||May 29, 2008||Jul 1, 2010||Tsuneo Nakata||Information processor|
|U.S. Classification||710/10, 707/999.201|
|International Classification||H04L29/06, G06F13/00, H04L29/08, G06F12/00, G06F17/30|
|Cooperative Classification||Y10S707/99952, H04L69/329, H04L67/06, H04L29/06, G06F17/30176|
|European Classification||G06F17/30F7Y1, H04L29/08N5, H04L29/06|
|Feb 13, 2002||AS||Assignment|
Owner name: LAPLINK, INC., WASHINGTON
Free format text: CHANGE OF NAME;ASSIGNOR:LAPLINK.COM, INC.;REEL/FRAME:012607/0912
Effective date: 20010612
|Apr 18, 2005||AS||Assignment|
Owner name: TIRO TRADING, LLC, WASHINGTON
Free format text: NUNC PRO TUNC ASSIGNMENT;ASSIGNOR:LAPLINK, INC.;REEL/FRAME:015908/0569
Effective date: 20050310
|Apr 19, 2005||AS||Assignment|
Owner name: LAPLINK SOFTWARE, INC., WASHINGTON
Free format text: NUNC PRO TUNC ASSIGNMENT;ASSIGNOR:TIRO TRADING, LLC;REEL/FRAME:015908/0745
Effective date: 20050302
|Aug 30, 2007||AS||Assignment|
Owner name: MICROSOFT CORPORATION, WASHINGTON
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:LAPLINK SOFTWARE, INC.;REEL/FRAME:019767/0279
Effective date: 20070629
|Jul 22, 2009||FPAY||Fee payment|
Year of fee payment: 12
|Dec 9, 2014||AS||Assignment|
Owner name: MICROSOFT TECHNOLOGY LICENSING, LLC, WASHINGTON
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:MICROSOFT CORPORATION;REEL/FRAME:034541/0001
Effective date: 20141014