Multi-process compression

1. A system for compressing a file having a plurality of records, the system comprising:

a splitter operable to partition the file into a first subfile and a second subfile, wherein each subfile comprises a subset of the records contained in the file;

a first compressor coupled to the splitter and operable to compress the first subfile to produce a first compressed subfile;

a second compressor coupled to the splitter and operable to compress the second subfile to produce a second compressed subfile; and

a combiner coupled to the compressors, the combiner operable to combine the compressed subfiles produced by the compressors to produce a compressed file.

2. The system of claim 1, further comprising a first microprocessor and a second microprocessor, wherein the first compressor executes on the first microprocessor and the second compressor executes on the second microprocessor.

3. The system of claim 1, wherein each compressor comprises a separate software process.

4. The system of claim 1, wherein the splitter is further operable to generate a subfile pointer for each subfile to specify the first record of the subfile.

5. The system of claim 1, wherein each compressor is further operable to set a subfile pointer forward to the beginning of a first record for an associated subfile, and further operable to compress a last record in the subfile if the associated subfile comprises at least a portion of the last record.

6. The system of claim 1, wherein each compressor is further operable to set a subfile pointer backwards to the beginning of a first record for an associated subfile, and further operable to compress a last record in the subfile that comprises a complete record.

7. A method performed on a computer for compressing a file having a plurality of records, the method comprising:

partitioning the file into a first subfile and a second subfile, wherein each subfile comprises a subset of the records contained in the file;

compressing the first subfile to produce a first compressed subfile;

compressing the second subfile to produce a second compressed subfile; and

combining the compressed subfiles to produce a compressed file.

8. The method of claim 7, wherein the steps of compressing the subfiles comprise:

providing a first compressor and a second compressor;

compressing the first subfile using the first compressor; and

compressing the second subfile using the second compressor.

9. The method of claim 7, wherein the steps of compressing the subfiles comprise:

spawning a first compressor and a second compressor as separate software processes;

compressing the first subfile using the first compressor; and

compressing the second subfile using the second compressor.

10. The method of claim 7, wherein the steps of compressing the subfiles further comprise:

setting a subfile pointer forward to the beginning of a first record for each subfile; and

compressing a last record in each subfile if the associated subfile comprises at least a portion of the last record.

11. The method of claim 7, wherein the steps of compressing the subfiles further comprise:

setting a subfile pointer backwards to the beginning of a first record for each subfile; and

compressing a last record in each subfile that comprises a complete record.

12. A compressor for compressing a file having a plurality of records with delimited fields, the compressor comprising:

a first buffer operable to store a first record of the file;

a second buffer operable to store a second record of the file; and

a processor coupled to the first buffer and the second buffer, the processor operable to compare, on a field-by-field basis, the first record to the second record, the processor operable to replace a field of the second record with a vertical repeat indicator if the field of the second record matches the corresponding field of the first record.

13. The compressor of claim 12, wherein the first record and the second record comprise adjacent records contained in the file.

14. The compressor of claim 12, wherein the processor is further operable to compare a first field of the second record with a second field of the second record, and to replace the first field with a horizontal repeat indicator if the first field matches the second field.

15. The compressor of claim 14, wherein the horizontal repeat indicator comprises:

the vertical repeat indicator; and

a number indicating the repetitions of the vertical repeat indicator.

16. The compressor of claim 12, wherein the processor is further operable to compare a first character of a field of the second record with a second character of the field, and to replace the first character with a horizontal repeat indicator if the first character matches the second character.

17. The compressor of claim 16, wherein the horizontal repeat indicator comprises:

the character; and

a number indicating the repetitions of the character.

18. The compressor of claim 12, wherein:

the file comprises a plurality of subfiles, each subfile comprising a subset of the records contained in the file; and

the processor is further operable to set a subfile pointer forward to the beginning of the first record for each subfile, and to compress a last record in each subfile if the associated subfile comprises at least a portion of the last record.

19. The compressor of claim 12, wherein:

the file comprises a plurality of subfiles, each subfile comprising a subset of the records contained in the file; and

the processor is further operable to set a subfile pointer backwards to the beginning of a first record for each subfile, and to compress a last record in each subfile that comprises a complete record.

20. A method for compressing a file having a plurality of records with delimited fields, the method comprising:

comparing, on a field-by-field basis, a first record of the file to a second record of the file; and

replacing a field of the second record with a vertical repeat indicator if the field of the second record matches the corresponding field of the first record.

21. The method of claim 20, wherein the first record and the second record comprise adjacent records contained in the file.

22. The method of claim 20, further comprising the steps of:

comparing a first field of the second record with a second field of the second record; and

replacing the first field with a horizontal repeat indicator if the first field matches the second field.

23. The method of claim 22, wherein the horizontal repeat indicator comprises:

the vertical repeat indicator; and

a number indicating the repetitions of the vertical repeat indicator.

24. The method of claim 20, further comprising the steps of:

comparing a first character of a field of the second record with a second character of the field; and

replacing the first character with a horizontal repeat indicator if the first character matches the second character.

25. The method of claim 24, wherein the horizontal repeat indicator comprises:

a character; and

a number indicating the repetitions of the character.

26. The method of claim 20, wherein the file comprises a plurality of subfiles, each subfile comprising a subset of the records contained in the file, and further comprising the steps of:

setting a subfile pointer forward to the beginning of a first record for each subfile; and

compressing a last record in each subfile if the associated subfile comprises at least a portion of the last record.

27. The method of claim 20 wherein the file comprises a plurality of subfiles, each subfile comprising a subset of the records contained in the file, and further comprising the steps of:

setting a subfile pointer backwards to the beginning of a first record for each subfile; and

compressing a last record in each subfile that comprises a complete record.

28. A method for decompressing a compressed file having a plurality of records with delimited fields, the method comprising:

determining, on a field-by-field basis, if a field of a compressed record comprises a vertical repeat indicator; and

writing the field to a buffer if the field of the compressed record is not a vertical repeat indicator.

29. The method of claim 28, further comprising writing a corresponding field of a previous decompressed record to the buffer if the field of the compressed record is a vertical repeat indicator.

30. The method of claim 29, wherein the decompressed record and the compressed record comprise adjacent records of the file.

31. The method of claim 28, further comprising:

determining, on a field-by-field basis, if a field of the compressed record comprises a horizontal repeat indicator having a character and a number indicating the repetitions of the character; and

writing to a second buffer the character of the horizontal repeat indicator according to the number indicating the repetitions of the character.

32. The method of claim 31, wherein the character of the horizontal repeat indicator comprises the vertical repeat indicator.