WO2008083914B1 - Method, apparatus, and computer program product for data upload in a computing system - Google Patents
Method, apparatus, and computer program product for data upload in a computing systemInfo
- Publication number
- WO2008083914B1 WO2008083914B1 PCT/EP2007/064431 EP2007064431W WO2008083914B1 WO 2008083914 B1 WO2008083914 B1 WO 2008083914B1 EP 2007064431 W EP2007064431 W EP 2007064431W WO 2008083914 B1 WO2008083914 B1 WO 2008083914B1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- file
- slot
- destination
- computer
- client
- Prior art date
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/40—Support for services or applications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
- G06F9/5072—Grid computing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/14—Multichannel or multilink protocols
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
- H04W4/06—Selective distribution of broadcast services, e.g. multimedia broadcast multicast service [MBMS]; Services to user groups; One-way selective calling services
Abstract
A method, system, and computer program product for simultaneous multi-channel upload of a file to one or more servers while ensuring data integrity. A validation scheme employs hashes to allow segments of the data file to be separately validated. Thus, if the upload process is interrupted or otherwise corrupted, segments of previously transferred data which have been transferred correctly may be validated, eliminating the need for re-transmission of that correctly transferred data. Preferably, a grid broker may be incorporated, allowing simultaneous multi-channel upload of data in a grid computing environment.
Claims
1. A computer-implemented method for uploading a data file over multiple channels from a client computer to at least one remote destination computer in a computing system, the method comprising the steps of: dividing the data file into a plurality of file slots and a plurality of file components, wherein a file component comprises one or more slots; transferring multiple file slots, each file slot operable to be transferred over one of the multiple channels, to upload the data file from client computer to the at least ooe destination computer; calculating, based upon the data file transferred to the destination computer, a destination-side file slot hash based upon the data in each file slot, a destination-side file component hash based upon the data in each file component, and a destination-side cumulative hash based upon the transferred data file; calculating a client-side cumulative hash based upon the data file resident on the client computer; comparing the destination-side and client-side cumulative hashes to verify integrity of the transferred file; and if the client-side and destination-side cumulative hashes fail to match, determining in which file slot and in which file component an error has occurred, such that uploading of the data file is operable to be re-started beginning with the file slot and file component containing the error.
2. The computer-implemented method of claim 1 , wherein the step of determining in which file slot and in which file component the error has occurred further comprises: comparing the destination-side file component hash and the destination-side file slot hash with the client-side file component hash and client-side file slot hash, respectively.
3. The computer-implemented method of claim 1 , further comprising the step of: generating a culled slot; merging, with the culled slot, transferred data adjacent to the culled slot; wherein calculation of the destination-side cumulative hash is performed using the data corresponding to the resulting culled slot
4. The computer-implemented method of claim 3, further comprising the step of: saving the destination-side cumulative hash state as metadata.
5. The computer-implemented method of claim 1 , further comprising the step of: associating an expirable lease with a slot
6. The computer-implemented method of claim 5, further comprising the steps of: determining whether the lease has expired; and in response to determining that the lease has expired, reclaiming the slot,
7. The computer-implemented method of claim 1 , further comprising the step of: generating a new slot by selecting a smallest file offset that is not represented by the culled slot or any existing slots.
8. The computer-implemented method of claim 1 , wherein the steps are operable to be implemented using multiple client-side intelligent agents equal in number to the multiple channels and at least one destination-side intelligent agent
9. An apparatus for uploading a data file over multiple channels from a client computer to at least one remote destination computer in a computing system, the apparatus comprising: means for dividing the data file into a plurality of file slots and a plurality of file components, wherein a file component comprises one or more slots; means for controlling transfer of multiple file slots, each file slot operable to be transferred over one of the multiple channels, to upload the data file from client computer to the at least one remote destination computer; means for calculating, based upon the data file transferred to the destination computer, a destination-side file slot hash based upon the data in each file slot, a destination-side file component hash based upon the data in each file component, and a destination-side cumulative hash based upon the transferred data file; 26
means for calculating a client-side cumulative hash based upon the data file resident on the client computer; means for comparing the destination-side and client-side cumulative hashes to verify integrity of the transferred file; and means for determining, if the client-side and destination-side cumulative hashes fail to match, in which file slot and in which file component the error has occurred, such that uploading of the data file is operable to be re-started beginning with the file slot and file component containing the error.
10. The apparatus of claim 9, wherein the means for determining in which file slot and in which file component the error has occurred further comprises: means for comparing the destination-side file component hash and the destination-side file slot hash with the client-side file component hash and client-side file slot hash, respectively.
11. The apparatus of claim 9, further comprising: means for generating a culled slot; means for merging, with the culled slot, transferred data adjacent to the culled slot; wherein calculation of the destination-side cumulative hash is performed using the data corresponding to the resulting culled slot.
12. The apparatus of claim 11 , further comprising: means for saving the destination-side cumulative hash state as metadata.
13. The apparatus of claim 9, further comprising: means for associating an expirable lease with a slot.
14. The apparatus of claim 13, further comprising means for determining whether the lease has expired; and means, responsive to a determination that the lease has expired, for reclaiming the slot. 27
15. The apparatus of claim 9, further comprising: means for generating a new slot by selecting a smallest file offset that is not represented by the culled slot or any existing slots.
16. The apparatus of claim 9, wherein the means for controlling transfer of multiple file slots comprises: multiple client-side intelligent agents equal in number to the multiple channels and at least one destination-side intelligent agent.
17. A computer program product comprising: a computer usable medium having computer readable program code for uploading a data file over multiple channels from a client computer to at least one remote destination computer, the computer program product comprising: first computer program code means for dividing the data file into a plurality of file slots and a plurality of file components, wherein a file component comprises one or more slots; second computer program code means for controlling transfer of multiple file slots, each file slot being transferred over one of the multiple channels, to upload the data file from client computer to the at least one destination computer; third computer program code means for calculating, based upon the data file transferred to the destination computer, a destination-side file slot hash based upon the data in each file slot, a destination-side file component hash based upon the data in each file component, and a destination-side cumulative hash; fourth computer program code means for calculating a client-side cumulative hash based upon the data file resident on the client computer, fifth computer program code means for comparing the destination-side and client-side cumulative hashes to verify integrity of the transferred file; and sixth computer program code means for determining, if the client-side and destination-side cumulative hashes fail to match, in which file slot and in which file component the error has occurred, such that uploading of the data file is operable to be re-started beginning with the file slot and file component containing the error. 28
18. The computer program, product of claim 17S further comprising: program code means for comparing the destination-side file component hash and the destination-side file slot hash with the client-side file component hash and client-side file slot hash, respectively.
19. The computer program product of claim 17 further comprising: program code means for generating a culled slot; program code means for merging, with the culled slot, transferred data adjacent to the culled slot; wherein calculation of the destination-side cumulative hash is operable to be performed using the data corresponding to the resulting culled slot.
20. The computer program product of claim 19, further comprising: program code means for saving the destination-side cumulative hash state as metadata.
21. The computer program product of claim 17, further comprising program code means for associating an expirable lease with a slot.
22. The computer program product of claim 21 , further comprising program code means for determinmg whether the lease has expired; and responsive to a determination that the lease has expired, reclaiming the slot
23. The computer program product of claim 17, further comprising program code means for generating a new slot by selecting a smallest file offset that is not represented by the culled slot or any existing slots.
24. The computer program product of claim 17, further comprising multiple client-side intelligent agents equal in number to the multiple channels and at least one destination-side intelligent agent.
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
DE602007006058T DE602007006058D1 (en) | 2007-01-12 | 2007-12-21 | Method, apparatus and computer program for uploading data in a data processing system |
EP07866303A EP2108228B1 (en) | 2007-01-12 | 2007-12-21 | Method, apparatus, and computer program for data upload in a computing system |
AT07866303T ATE465589T1 (en) | 2007-01-12 | 2007-12-21 | METHOD, APPARATUS AND COMPUTER PROGRAM FOR UPLOADING DATA INTO A DATA PROCESSING SYSTEM |
KR1020097012918A KR101063556B1 (en) | 2007-01-12 | 2007-12-21 | Methods, devices, and computer program products for uploading data in computing systems |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/622,711 | 2007-01-12 | ||
US11/622,711 US8055902B2 (en) | 2007-01-12 | 2007-01-12 | Method, system, and computer program product for data upload in a computing system |
Publications (2)
Publication Number | Publication Date |
---|---|
WO2008083914A1 WO2008083914A1 (en) | 2008-07-17 |
WO2008083914B1 true WO2008083914B1 (en) | 2008-08-28 |
Family
ID=39332072
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/EP2007/064431 WO2008083914A1 (en) | 2007-01-12 | 2007-12-21 | Method, apparatus, and computer program product for data upload in a computing system |
Country Status (6)
Country | Link |
---|---|
US (1) | US8055902B2 (en) |
EP (1) | EP2108228B1 (en) |
KR (1) | KR101063556B1 (en) |
AT (1) | ATE465589T1 (en) |
DE (1) | DE602007006058D1 (en) |
WO (1) | WO2008083914A1 (en) |
Families Citing this family (16)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20100121828A1 (en) * | 2008-11-11 | 2010-05-13 | You Wang | Resource constraint aware network file system |
JP5534031B2 (en) * | 2010-12-15 | 2014-06-25 | 富士通株式会社 | Data migration program, computer and data migration method |
US20120246205A1 (en) * | 2011-03-23 | 2012-09-27 | Hitachi, Ltd. | Efficient data storage method for multiple file contents |
CN102833294B (en) | 2011-06-17 | 2015-05-20 | 阿里巴巴集团控股有限公司 | File processing method and system based on cloud storage, and server cluster system |
US8635270B2 (en) * | 2011-09-19 | 2014-01-21 | Verizon Patent And Licensing Inc. | Thread mechanism for media and metadata upload |
JP5857684B2 (en) * | 2011-11-30 | 2016-02-10 | ブラザー工業株式会社 | Communication device |
US9160697B2 (en) * | 2012-01-01 | 2015-10-13 | Qualcomm Incorporated | Data delivery optimization |
CN103309768B (en) * | 2012-03-16 | 2015-03-11 | 腾讯科技(深圳)有限公司 | Method and device for repairing system files |
JP5987445B2 (en) * | 2012-04-20 | 2016-09-07 | 富士通株式会社 | COMMUNICATION CONTROL DEVICE, METHOD, PROGRAM, AND SYSTEM |
CA2881567C (en) * | 2012-08-08 | 2023-10-17 | Amazon Technologies, Inc. | Archival data storage system and method |
US20140143201A1 (en) * | 2012-11-20 | 2014-05-22 | International Business Machines Corporation | Dynamic content file synchronization |
CN103248711B (en) * | 2013-05-23 | 2016-08-24 | 华为技术有限公司 | A kind of method of files passe and server |
CN107403016A (en) * | 2017-08-08 | 2017-11-28 | 郑州云海信息技术有限公司 | A kind of file uploading method and device |
US11768954B2 (en) | 2020-06-16 | 2023-09-26 | Capital One Services, Llc | System, method and computer-accessible medium for capturing data changes |
KR102246581B1 (en) * | 2021-01-04 | 2021-04-29 | 이선웅 | Method for uploading file via cloud computing environment, and proxy server for executing the same |
FI129641B (en) * | 2021-01-27 | 2022-06-15 | Nokia Technologies Oy | Receiving segments of system information block message |
Family Cites Families (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP1431864B2 (en) * | 1995-02-13 | 2012-08-22 | Intertrust Technologies Corporation | Systems and methods for secure transaction management and electronic rights protection |
US6049892C1 (en) | 1997-02-24 | 2002-06-04 | Ethos Software Corp | Process and apparatus for downloading data from a server computer to a client computer |
US6477522B1 (en) | 1999-06-10 | 2002-11-05 | Gateway, Inc. | Dynamic performance based server selection |
US6339785B1 (en) | 1999-11-24 | 2002-01-15 | Idan Feigenbaum | Multi-server file download |
US20030074465A1 (en) | 2001-10-15 | 2003-04-17 | Zhicheng Tang | ADSL downloading with priority transmit queue |
US7716659B2 (en) | 2001-10-23 | 2010-05-11 | Pitney Bowes Inc. | Remote monitoring and software distribution system for servicing inserter systems |
US8261059B2 (en) | 2001-10-25 | 2012-09-04 | Verizon Business Global Llc | Secure file transfer and secure file transfer protocol |
US7555559B2 (en) | 2003-02-28 | 2009-06-30 | Onion Networks, KK | Parallel data transfer over multiple channels with data order prioritization |
US7627617B2 (en) * | 2004-02-11 | 2009-12-01 | Storage Technology Corporation | Clustered hierarchical file services |
US7631098B2 (en) | 2004-06-08 | 2009-12-08 | International Business Machines Corporation | Method, system and program product for optimized concurrent data download within a grid computing environment |
JP4337675B2 (en) * | 2004-07-23 | 2009-09-30 | ソニー株式会社 | Cryptographic processing apparatus and cryptographic processing method |
US8103870B2 (en) * | 2006-09-12 | 2012-01-24 | Foleeo, Inc. | Hive-based peer-to-peer network |
-
2007
- 2007-01-12 US US11/622,711 patent/US8055902B2/en not_active Expired - Fee Related
- 2007-12-21 WO PCT/EP2007/064431 patent/WO2008083914A1/en active Application Filing
- 2007-12-21 DE DE602007006058T patent/DE602007006058D1/en active Active
- 2007-12-21 AT AT07866303T patent/ATE465589T1/en not_active IP Right Cessation
- 2007-12-21 EP EP07866303A patent/EP2108228B1/en not_active Not-in-force
- 2007-12-21 KR KR1020097012918A patent/KR101063556B1/en not_active IP Right Cessation
Also Published As
Publication number | Publication date |
---|---|
WO2008083914A1 (en) | 2008-07-17 |
US20080172392A1 (en) | 2008-07-17 |
DE602007006058D1 (en) | 2010-06-02 |
KR20090097165A (en) | 2009-09-15 |
US8055902B2 (en) | 2011-11-08 |
ATE465589T1 (en) | 2010-05-15 |
KR101063556B1 (en) | 2011-09-07 |
EP2108228B1 (en) | 2010-04-21 |
EP2108228A1 (en) | 2009-10-14 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2008083914B1 (en) | Method, apparatus, and computer program product for data upload in a computing system | |
US7660956B1 (en) | Save set bundling for staging | |
CN106570086B (en) | Data migration system and data migration method | |
US7895501B2 (en) | Method for auditing data integrity in a high availability database | |
US9378155B2 (en) | Method for processing and verifying remote dynamic data, system using the same, and computer-readable medium | |
US9235593B2 (en) | Transmitting filesystem changes over a network | |
CN109799948B (en) | Data storage method and device | |
US20090083732A1 (en) | Creation and deployment of distributed, extensible applications | |
WO2007106569A3 (en) | Systems and methods for controlling error reporting and resolution | |
WO2017039803A1 (en) | Systems and methods for verification of code resiliency for data storage | |
US10146633B2 (en) | Data recovery from multiple data backup technologies | |
WO2020035092A3 (en) | Blockchain data storage based on error correction code for permissioned blockchain network | |
US20130166862A1 (en) | Efficient backup replication | |
US9983948B2 (en) | Caching of backup chunks | |
US9489254B1 (en) | Verification of erasure encoded fragments | |
US7849355B2 (en) | Distributed object sharing system and method thereof | |
US7770065B2 (en) | File data restoring system and method of computer operating system and software thereof | |
US7765371B2 (en) | Method and apparatus for full backups in advance | |
CN107316176A (en) | The control method and device of declaration form processing | |
JP7139347B2 (en) | Method for Partial Update of Data Content in Scattered Storage Networks | |
US9552254B1 (en) | Verification of erasure encoded fragments | |
US20120017075A1 (en) | Registry key federation systems and methods | |
CN105243077A (en) | Network based unlimited-length file transmission method and system | |
US20130297283A1 (en) | Information processing device, information processing method, and program | |
US10489247B2 (en) | Generating time-ordered globally unique revision numbers |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 07866303 Country of ref document: EP Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 1020097012918 Country of ref document: KR |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2007866303 Country of ref document: EP |