Method for partitioning a block of data into subblocks and for storing and ...

1. A method for organizing a block b of digital data for storage, communication, or comparison, comprising the step of:

partitioning said block b into a plurality of subblocks at at least one position k.vertline.k+1 within said block,

for which b[k-A+1 . . . k+B] satisfies a predetermined constraint, and

wherein A and B are natural numbers.

2. The method of claim 1, wherein the constraint comprises the hash of at least a portion of b[k-A+1 . . . k+B].

3. The method of claim 1, further comprising the step of:

locating the nearest subblock boundary on a side of a position p.vertline.p+1 within said block, said locating step comprising the step of:

evaluating whether said predetermined constraint is satisfied at each position k.vertline.k+1 for increasing or decreasing k,

wherein k starts with the value p.

4. The method of claim 1, wherein at least one bound is imposed on the size of at least one of said plurality of subblocks.

5. The method of claim 1, wherein additional subblocks are formed from at least one group of subblocks.

6. The method of claim 1, wherein an additional hierarchy of subblocks is formed from at least one group of contiguous subblocks.

7. The method of claim 1, further comprising the step of:

calculating the hash of each of at least one of said plurality of subblocks.

8. The method of claim 1, further comprising the step of:

forming a projection of said block, being an ordered or unordered collection of elements, wherein each element consists of a subblock, an identity of a subblock, or a reference of a subblock.

9. The method of claim 1, wherein said subblocks are compared by comparing the hashes of said subblocks.

10. The method of claim 1, wherein subsets of identical subblocks within a group of one or more subblocks are found by inserting each subblock, an identity of each subblock, a reference of each subblock, or a hash of each subblock into a data structure.

11. A method for comparing one or more blocks, comprising the steps of:

organizing a block b of digital data for the purpose of comparison, comprising the step of:

partitioning said block b into a plurality of subblocks at at least one position k.vertline.k+1 within said block;

for which b[k-A+1 . . . k+B] satisfies a predetermined constraint; and

wherein A and B are natural numbers,

forming a projection of each said block, being a collection of elements, wherein each element comprises a selected one of a subblock, an identity of a subblock, and a reference of a subblock, and

comparing the elements of said projections of said blocks.

12. A method for representing one or more blocks comprising a collection of subblocks and block representatives which are mapped to lists of entries which identify subblocks; said method comprising the step of modifying one of said blocks including the steps of:

partitioning said block into a plurality of subblocks at at least one position k.vertline.k+1 within said block, for which b[k-A+1 . . . k+B] satisfies a predetermined constraint, and wherein A and B are natural numbers,

adding to said collection of subblocks zero or more subblocks which are not already in said collection, and

updating said subblock list associated with said modified block.

13. A method for representing one or more blocks comprising a collection of subblocks and block representatives which are mapped to lists of entries which identify subblocks; said method comprising the step of modifying one of said blocks including the steps of:

partitioning said block into a plurality of subblocks at at least one position k.vertline.k+1 within said block, for which b[k-A+1 . . . k+B] satisfies a predetermined constraint, and wherein A and B are natural numbers,

removing from said collection of subblocks zero or more subblocks, and

updating said subblock list associated with said modified block.

14. A method for representing one or more blocks comprising a collection of subblocks and block representatives which are mapped to lists of entries which identify subblocks; said method comprising the step of modifying one of said blocks including the steps of:

partitioning said block into a plurality of subblocks at at least one position k.vertline.k+1 within said block, for which b[k-A+1 . . . k+B] satisfies a predetermined constraint, and wherein A and B are natural numbers,

adding to said collection of subblocks zero or more subblocks that are not already in the collection,

removing from said collection of subblocks zero or more subblocks, and

updating said subblock list associated with said modified block.

15. A method for an entity E1 to communicate a block X to E2 where E1 possesses the knowledge that E2 possesses a group of Y subblocks Y.sub.1 . . . Y.sub.m, comprising the steps of:

partitioning said block X into a plurality of subblocks X.sub.1 . . . X.sub.n at at least one position k.vertline.k+1 within said block, for which X[k-A+1 . . . k+B] satisfies a predetermined constraint, and wherein A and B are natural numbers, and

transmitting from E1 to E2 the contents of zero or more subblocks in X.sub.1 and the remaining subblocks as references to subblocks in Y.sub.1 . . . Y.sub.m, and to subblocks transmitted.

16. A method for an entity E1 to communicate one or more subblocks of a group X of subblocks X.sub.1 . . . X.sub.n to E2 where E1 possesses the knowledge that E2 possesses a block Y, comprising the steps of:

partitioning said block Y into a plurality of subblocks Y.sub.1 . . . Y.sub.m at at least one position k.vertline.k+1 within said block, for which Y[k-A+1 . . . k+B] satisfies a predetermined constraint, and wherein A and B are natural numbers, and

transmitting from E1 to E2 the contents of zero or more subblocks in X, and the remaining subblocks as references to subblocks in Y, and to subblocks already transmitted.

17. A method for an entity E1 to communicate a block X to E2 where E1 possesses the knowledge that E2 possesses a block Y, comprising the steps of:

partitioning said block X into a plurality of subblocks X.sub.1 . . . X.sub.n at at least one position k.vertline.k+1 within said block, for which X[k-A+1 . . . k+B] satisfies a predetermined constraint, and wherein A and B are natural numbers,

partitioning said block Y into a plurality of subblocks Y.sub.1 . . . Y.sub.m at at least one position k.vertline.k+1 within said block, for which Y[k-A+1 . . . k+B] satisfies a predetermined constraint, and wherein A and B are natural numbers, and

transmitting from E1 to E2 the contents of zero or more subblocks in X, and the remaining subblocks as references to subblocks in Y, and to subblocks already transmitted.

18. A method for constructing a block D from a block X and a group Y of subblocks Y.sub.1 . . . Y.sub.m such that X can be constructed from Y and D, comprising the steps of:

partitioning said block X into a plurality of subblocks X.sub.1 . . . X.sub.n at at least one position k.vertline.k+1 within said block, for which X[k-A+1 . . . k+B] satisfies a predetermined constraint, and wherein A and B are natural numbers, and

constructing D from a selected at least one of:

the contents of zero or more subblocks in X,

references to zero or more subblocks in Y, and

references to zero or more subblocks in D.

19. A method for constructing a block D from a group X of subblocks X.sub.1 . . . X.sub.n and a block Y such that X can be constructed from Y and D, comprising the steps of:

partitioning said block Y into a plurality of subblocks Y.sub.1 . . . Y.sub.m at at least one position k.vertline.k+1 within said block, for which Y[k-A+1 . . . k+B] satisfies a predetermined constraint, and wherein A and B are natural numbers, and

constructing D from a selected at least one of:

the contents of zero or more subblocks in X,

references to zero or more subblocks in Y, and

references to zero of more subblocks in D.

20. A method for constructing a block D from a block X and a block Y such that X can be constructed from Y and D, comprising the steps of:

partitioning said block X into a plurality of subblocks X.sub.1 . . . X.sub.n at at least one position k.vertline.k+1 within said block, for which X[k-A+1 . . . k+B] satisfies a predetermined constraint, and wherein A and B are natural numbers,

partitioning said block Y into a plurality of subblocks Y.sub.1 . . . Y.sub.m at at least one position k.vertline.k+1 within said block, for which Y[k-A+1 . . . k+B] satisfies a predetermined constraint, and wherein A and B are natural numbers, and

constructing D from a selected at least one of:

the contents of zero or more in X,

references to zero or more subblocks in Y, and

references to zero or more subblocks in D.

21. A method for constructing a block D from a block X and a projection Y said projection comprising a collection of elements wherein said elements comprises a subblock in Y, an identity of a subblock in Y, or a reference of a subblock in Y, such that X can be constructed from Y and D, comprising the steps of:

partitioning said block X into a plurality of subblocks X.sub.1 . . . X.sub.n at at least one position k.vertline.k+1 within said block, for which X[k-A+1 . . . k+B] satisfies a predetermined constraint, and wherein A and B are natural numbers, and

constructing D from a selected at least one of:

the contents of zero or more in X,

references to zero or more subblocks in Y, and

references to zero or more subblocks in D.

22. A method for constructing a block X from a block Y and a block D, comprising the steps of:

partitioning said block Y into a plurality of subblocks Y.sub.1 . . . Y.sub.m at at least one position k.vertline.k+1 within said block, for which Y[k-A+1 . . . k+B] satisfies a predetermined constraint, and wherein A and B are natural numbers, and

constructing X from D and Y by constructing the subblocks of X based on a selected at least one of:

subblocks contained within D,

references in D to subblocks in Y, and

references to D to subblocks in D.

23. A method for constructing a group X of subblocks X.sub.1 . . . X.sub.n from a block Y and a block D, comprising the steps of:

partitioning said block Y into a plurality of subblocks Y.sub.1 . . . Y.sub.m at at least one position k.vertline.k+1 within said block, for which Y[k-A+1 . . . k+b] satisfies a predetermined constraint, and wherein A and B are natural numbers, and

constructing X.sub.1 . . . X.sub.n from D and Y based on a selected at least one of:

subblocks contained within D,

references in D to subblocks in Y, and

references to D to subblocks in D.

24. A method for communicating a data block X from one entity E1 to another entity E2, comprising the steps of:

partitioning said block X into a plurality of subblocks X.sub.1 . . . X.sub.n at at least one position k.vertline.k+1 within said block, for which X[k-A+1 . . . k+B] satisfies a predetermined constraint, and wherein A and B are natural numbers,

transmitting from E1 to E2 an identity of at least one subblock,

transmitting from E2 to E1 information communicating the presence or absence of subblocks at E2, and

transmitting from E1 to E2 at least the subblocks identified as not being present at E2.

25. A method for communicating a block X from one entity E1 to another entity E2, comprising the steps of:

partitioning said block X into a plurality of subblocks X.sub.1 . . . X.sub.n at at least one position k.vertline.k+1 within said block, for which X[k-A+1 . . . k+B] satisfies a predetermined constraint, and wherein A and B are natural numbers,

transmitting from E2 to E1 information communicating the presence or absence at E2 of members of a group Y of subblocks Y.sub.1 . . . Y.sub.m, and

transmitting from E1 to E2 the contents of zero or more subblocks in X, and the remaining subblocks as references to subblocks in Y.sub.1 . . . Y.sub.m and to subblocks already transmitted.

26. A method for an entity E2 to communicate to an entity E1 the fact that E2 possesses a block Y, comprising the steps of:

partitioning said block Y into a plurality of subblocks Y.sub.1 . . . Y.sub.m at at least one position k.vertline.k+1 within said block, for which Y[k-A+1 . . . k+B] satisfies a predetermined constraint, and wherein A and B are natural numbers, and

transmitting from E2 to E1 references of the subblocks Y.sub.1 . . . Y.sub.m.

27. A method for an entity E1 to communicate a subblock X.sub.1 to an entity E2, comprising the steps of:

partitioning said block X into a plurality of subblocks X.sub.1 . . . X.sub.n at at least one position k.vertline.k+1 within said block, for which X[k-A+1 . . . k+B] satisfies a predetermined constraint, and wherein A and B are natural numbers,

transmitting from E2 to E1 an identity of X.sub.i,

transmitting X.sub.i from E1 to E2.

28. An apparatus for organizing a block b of digital data for storage, communication, or comparison, comprising

means for partitioning said block b into a plurality of subblocks at at least one position k.vertline.k+1 within said block, for which b[k-A+1 . . . k+B] satisfies a predetermined constraint, and wherein A and B are natural numbers.

29. The apparatus of claim 28, in which the constraint comprises the hash of some or all of b[k-A+1 . . . k+B].

30. The apparatus of claim 28, further comprising

means for locating the nearest subblock boundary on a side of a position p.vertline.p+1 within said block, said means for locating comprising:

means for evaluating whether said predetermined constraint is satisfied at each position k.vertline.k+1 for increasing or decreasing k,

wherein k starts with the value p.