CA2065578C - Packet-based data compression method - Google Patents
Packet-based data compression methodInfo
- Publication number
- CA2065578C CA2065578C CA002065578A CA2065578A CA2065578C CA 2065578 C CA2065578 C CA 2065578C CA 002065578 A CA002065578 A CA 002065578A CA 2065578 A CA2065578 A CA 2065578A CA 2065578 C CA2065578 C CA 2065578C
- Authority
- CA
- Canada
- Prior art keywords
- packet
- data
- static
- user
- fields
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Lifetime
Links
Classifications
-
- H—ELECTRICITY
- H03—ELECTRONIC CIRCUITRY
- H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
- H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
- H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
- H03M7/3084—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction using adaptive string matching, e.g. the Lempel-Ziv method
- H03M7/3088—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction using adaptive string matching, e.g. the Lempel-Ziv method employing the use of a dictionary, e.g. LZ78
-
- H—ELECTRICITY
- H03—ELECTRONIC CIRCUITRY
- H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
- H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
- H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/04—Protocols for data compression, e.g. ROHC
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
- H04L9/40—Network security protocols
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/22—Parsing or analysis of headers
Abstract
A data processing system is described that employs data packets which include at least static and dynamic fields, the static fields containing information that often remains constant during a multi-packet communication interval and the dynamic fields containing information that changes for each packet.
Many packets also include a user-data fields. A
compression method is described which comprises:
reformatting each data packet by associating its static fields with a first packet region and its dynamic fields with a second packet region. The process then assembles a static table that includes static information from at least an initial data packet's first packet region. It then identifies static field information in a subsequent data packet's first packet region that is common to the information in the static table. Such common information is encoded so as to reduce its data length. The common static information is then replaced in the modified data packet with the encoded common static information and the modified data packet is then transmitted. A
similar action occurs with respect to user-data information. A single dictionary table is created for all packet headers, while separate dictionary tables are created for each user-data portion of a packet-type experienced in the communication network thereby enabling better compression.
Many packets also include a user-data fields. A
compression method is described which comprises:
reformatting each data packet by associating its static fields with a first packet region and its dynamic fields with a second packet region. The process then assembles a static table that includes static information from at least an initial data packet's first packet region. It then identifies static field information in a subsequent data packet's first packet region that is common to the information in the static table. Such common information is encoded so as to reduce its data length. The common static information is then replaced in the modified data packet with the encoded common static information and the modified data packet is then transmitted. A
similar action occurs with respect to user-data information. A single dictionary table is created for all packet headers, while separate dictionary tables are created for each user-data portion of a packet-type experienced in the communication network thereby enabling better compression.
Description
-- 20~578' P~CKET-BASED DATA COMPRESSION METHOD
FIELD OF THE lNV~N~l~lON
This invention relates to data compression techniques, and more particularly, to a method for transmission of data packets between local area networks, wherein information in the packets is compressed prior to transmission.
BACKGROUND OF THE INVEN~ION
iO
Data transferred over communication links between commercial computer systems generally contains significant redundancy. A number of mechanisms and procedures exist for lessening the redundancy and for creating substantially more efficient use of the transmission bandwidth. The term "data compression"
refers to any process that converts data in a first given format into a second format having fewer bits than the original. Data compression systems are~
particularly effective if the original data contains substantial redundancy, such as symbols or strings of symbols which repetitively appear with high frequency.
Preferred data compression methods are transparent in that the application computer programmer is not aware of the existence of the compression method, except in system performance manifestations. As a result, decompressed data is an exact replica of the input data and the compression apparatus is given no special 2 ~ 7 ~
program information. So long as the transmission protocols are constant in the c~; ication network, transparent compression can be readily accomplished.
However, once various protocols and data formats find their way into a network, effective data compression bec ?S much more complex.
Such systems can be found today in wide area networks (WAN's), which interconnect pluralities of Local Area Networks (LAN's). In general, internal LAN
interconnections occur over wide bandwidth, hard-wired or optical interconnects that alleviate the requirements for data compression. By contrast, most WAN's employ the telephone network for LAN
interconnection purposes, and, as a result, are significantly bandwidth-limited.
There are a number of general purpose data compression procedures described in the prior art. A popular compression method, known as "Huffman" encoding translates fixed-size pieces of input data into variable-length symbols. The procedure assigns codes to input symbols such that each code length, in bits, is approximately log2(symbol probability), where symbol probability is the relative frequency of occurrence of a given symbol, expressed as a probability. Huffman encoding exhibits a number of limitations. The bit-run size of input symbols is limited by the size of the translation table needed for compression. The decompression process is complex, and it is also necessary to know the frequency distribution for the group of possible input symbols.
2 ~ 7 ~
A further type of encoding is known as "run-length"
encodinq and causes sequences of identical characters to be encoded as a count field appended to an identifier of the repeated character. While this approach is effective in graphical images, it has virtually no value in text and has moderate value for data files.
Recently, a method termed "adaptive" compression has appeared and has become, in various configurations, widely used. Algorithms for adaptive compression have been published by J. Ziv and A. Lempel, in "A
Universal Algorithm For Sequential Data Compression", IEEE Transactions, Information Theory, Vol. IT-23, No.
FIELD OF THE lNV~N~l~lON
This invention relates to data compression techniques, and more particularly, to a method for transmission of data packets between local area networks, wherein information in the packets is compressed prior to transmission.
BACKGROUND OF THE INVEN~ION
iO
Data transferred over communication links between commercial computer systems generally contains significant redundancy. A number of mechanisms and procedures exist for lessening the redundancy and for creating substantially more efficient use of the transmission bandwidth. The term "data compression"
refers to any process that converts data in a first given format into a second format having fewer bits than the original. Data compression systems are~
particularly effective if the original data contains substantial redundancy, such as symbols or strings of symbols which repetitively appear with high frequency.
Preferred data compression methods are transparent in that the application computer programmer is not aware of the existence of the compression method, except in system performance manifestations. As a result, decompressed data is an exact replica of the input data and the compression apparatus is given no special 2 ~ 7 ~
program information. So long as the transmission protocols are constant in the c~; ication network, transparent compression can be readily accomplished.
However, once various protocols and data formats find their way into a network, effective data compression bec ?S much more complex.
Such systems can be found today in wide area networks (WAN's), which interconnect pluralities of Local Area Networks (LAN's). In general, internal LAN
interconnections occur over wide bandwidth, hard-wired or optical interconnects that alleviate the requirements for data compression. By contrast, most WAN's employ the telephone network for LAN
interconnection purposes, and, as a result, are significantly bandwidth-limited.
There are a number of general purpose data compression procedures described in the prior art. A popular compression method, known as "Huffman" encoding translates fixed-size pieces of input data into variable-length symbols. The procedure assigns codes to input symbols such that each code length, in bits, is approximately log2(symbol probability), where symbol probability is the relative frequency of occurrence of a given symbol, expressed as a probability. Huffman encoding exhibits a number of limitations. The bit-run size of input symbols is limited by the size of the translation table needed for compression. The decompression process is complex, and it is also necessary to know the frequency distribution for the group of possible input symbols.
2 ~ 7 ~
A further type of encoding is known as "run-length"
encodinq and causes sequences of identical characters to be encoded as a count field appended to an identifier of the repeated character. While this approach is effective in graphical images, it has virtually no value in text and has moderate value for data files.
Recently, a method termed "adaptive" compression has appeared and has become, in various configurations, widely used. Algorithms for adaptive compression have been published by J. Ziv and A. Lempel, in "A
Universal Algorithm For Sequential Data Compression", IEEE Transactions, Information Theory, Vol. IT-23, No.
3, May, 1977, pp. 337-343 and in "Compression of Individual Sequences Via Variable-Rate Coding", IEEE
Transactions on Information Theory, Vol. IT-24, No. 5, September, 1978, pp. 5306.
The Lempel-Ziv algorithm converts variable length strings of input symbols into fixed-length (or predictable length) codes. The symbol strings are selected so that all have almost equal probability of occurrence. Consequently, strings of frequently occurring symbols will contain more symbols than a string having infrequent symbols. This form of compression is effective at exploiting character frequency redundancy, character repetitions, and high usage pattern redundancy.
One of the first algorithms published by Lempel-Ziv (typically referenced to as LZ77) maintains a history buffer of the last N characters from the input data stream (typically 4,096) and encodes the output data stream as a sequence of symbols. If the character string is not found in the history buffer, it is encoded as a zero, followed by the unencoded eight bit character, resulting in a nine bit code. "Unencoded"
in this sense means the eight bit binary character which corresponds to the alpha-numeric character. If a character or stream of characters is found in the buffer, the stream is encoded as a 1, followed by an index and length in the dictionary. This thereby enables the receiving end to reconstruct, from its copy of the buffer, the transmitted data.
More recently, a modification to a subsequent Lempel-Ziv data compression technique (LZ 78) published by T. Welch, has become known as the LZW
algorithm. The LZW algorithm converts strings of varying lengths from an input data stream to fixed-length or predictable length codes, typically 12 bits in length. The premise of the algorithm is that frequently occurring strings contain more characters than infrequently occurring strings.
Initially an LZW dictionary or code table is empty, except for the first 256 character positions which contain basic alpha-numeric single character entries.
A new entry is created whenever a previously unseen string is encountered. The compressor searches the input stream to determine the longest match to a string stored in the dictionary. Each stored string comprises a prefix string and an extension chara¢ter.
Each string has a code signal associated with it. A
string is stored in the string table by, at least implicitly, storing the code signal for the string.
When a longest match between an input data character 7 ~
stream and a stored stream is determined, the code signal for the longest match is transmitted as the "compressed" code signal and a new string is stored in the string table. The prefix of the new string is the longest match of string characters and the suffix is an extension character which is the next data character from the input data that resulted in the longest match. Thus, as each compression occurs, the string lengths are increased by the addition of the extension character. Additional details of this algorithm can be found in U.S. Patent 4,558,302 to T.A. Welch, and in an article by Welch entitled "A
Technique For High-Perfol ~nce Data Compression" IEEE
Computer, June 1984, pp. 8-19.
While the LZW data compression algorithm is widely used, it does exhibit a number of shortcomings. For instance, during early stages in the construction of the dictionary, many data fragments (i.e. character lengths of 2, 3, or 4) will occupy large parts of the available dictionary storage. Thus, the amounts of compression available will be limited by the available strings. Often, in lieu of compressing the data, it will actually e~pAnd the data being transmitted. For instance, when only a single character is found to match, as will be the case in the early stages of dictionary construction, the outputting of a 12 bit code for an 8 bit input character will result in a 50%
increase in data.
Accordingly, it is an object of this invention to provide a data compression method particularly adapted for use with packetized data.
2a6~
It is another object of this invention to provide an improved data compression method which avoids the accumulation of short data strings in the compression dictionary.
s It is still another object of this invention to provide a data compression method that accommodates a plurality of protocols employing different type packets.
Yet another object of this invention is to provide an altered method of operation for the LZW compression algorithm that enables the attainment of improved compression results.
SUMMARY OF THE lNvrNl~lON
A data processing system is described that employs data packets which include at least static and dynamic fields, the static fields containing information that often remains constant during a multi-packet communication interval, and the dynamic fields containing information that may change for each packet. Many packets also include user-data fields.
A compression method is described which comprises:
reformatting each data packet by associating its static fields with a first packet region and its dynamic fields with a second packet region. The process then assembles a static table that includes static information from at least an initial data packet's first packet region. It then identifies static field information in a subsequent data packet's first packet region that is common to the information in the static table. Such common information is 2 ~
encoded so as to reduce its data length. The common static information is then replaced in the modified data packet with the encoded common static information and the modified data packet is then transmitted. A
similar action occurs with respect to user-data information. A single dictionary table is created for all packet headers, while separate dictionary tables are created for each user-data portion of a packet-type experienced in the communication network, thereby enabling better compression.
DESCRIPTION OF THE DRAWINGS
Fig. 1 is a block diagram of a LAN to LAN
communication network over a wide area network (WAN) link.
Fig. 2 is a software/hardware block diagram of the LANBRIDGE processor and modem portions of a WAN link.
Fig. 3 is a block diagram showing major hardware components of a LANBRIDGE processor.
Fig. 4 is a TCP/IP over Ethernet packet format.
Fig. 5 i~ the packet format of Fig. 4 which has been reformatted in accordance with the invention.
Fig. 6 is a list of compression tables employed to compress the data in the reformatted packet of Fig. 5.
Figs. 7-9 illustrate a high level block diagram describing the method of the invention.
- 2 ~
Fig. lO is a depiction of a LAT packet showing byte numbers plotted along the left side of the packet depiction and the data indicated within the packet bytes.
Fig. 11 is the depiction of Fig. 10 after the LAT
packet has been truncated.
Fig. 12 is a depiction of the truncated LAT packet of Fig. 11 after certain bytes have been "swapped" in their positions within the packet's header structure.
Figs. 13 and 14 illustrate a high level flow diagram describing a method for data compressing the swapped LAT packet header structure.
DE~ATT~n DESCRIPTION OF THE lNV~-~ ~ lON
Referring first to Fig. 1, a pair of LAN's A and B are shown, one located in Boston and one in Ottawa. Each LAN has att~ched thereto, various devices which are well known in the art. As aforestated, there is little or no need for data compression within each of LANs A or B. On the other hand, when data is to be transmitted from LAN A to LAN B, it will first pass through LANBRIDGE 10, where the packet data appearing on LAN A is au lessed in accordance with the invention. Such encoded data is then transmitted by modem 12 over WAN link 14 to modem 16. The received data is decv ~-essed by LANBRIDGE 18, and the packet structures appearing at the input to LANBRIDGE 10 are reconstructed and emplaced on LAN B.
2~3~Ps ~
In Fig. 2, a combined hardware/software block diagram of LANBRIDGE 10 is shown. Data packets appearing on LAN A are received by LAN interface devices 20 and passed into a random access memory (RAM) within 5 LANBRIDGE processor 10. Within LANBRIDGE processor 10 is software 22 which, in addition to performing routing and other functions, also performs reformatting and data compression functions.
A high level block diagram of LANBRIDGE processor 10 is shown in Fig. 3. A central processing unit 30 forms the heart of LANBRIDGE processor 10 and communicates with other elements of the system via bus 32. A LAN interface 20 is connected to bus 32, as is WAN interface 34 (provides a gateway for data to and from modem 12). An electrically alterable, programmable read only memory 36 and a RAM 38 provide storage functions for CPU 30. Within RAM 38 are a plurality of compression tables that are employed in the operation of the invention.
Returning to Fig. 2, bridge software 22 within LANBRIDGE processor 10 contains portions which first reformat each packet received from LAN A, and then function to compress data in the reformatted packet in accordance with data stored in the compression tables in RAM 38. The compressed data is then transmitted, through WAN interface 34 and onto the wide area network.
For the remainder of this portion of the description, it will be assumed that LANBRIDGE processor 10 has received a TCP/IP packet from LAN A. It will be further assumed that it is employing an Ethernet packet format. Such a format is shown in Fig. 4. In brief, the Transmission Control Protocol/Internet Protocol (TCP/IP) is a well known and widely used protocol. At a high level, it includes four protocols, a network access protocol (NAP), an internet protocol (IP), a transmission control protocol (TCP), and a process application protocol (PAP). Subsidiary to the internet protocol, there can be other protocols besides TCP, for instance, an internet control message protocol (ICMP), a user datagram protocol (UDP), etc. It is to be understood that this invention is equally applicable to other protocols whose packet formats contain the classes of data to be hereinafter described, (i.e. local area transport protocol (LAT)) and it is in no way limited to the TCP/IP format.
In ~ig. 4, the fields present in a TCP/IP packet are shown, with the number of bytes in each indicated to the left of the Fig. The information contained within each of the fields is shown within each field's boundary. Certain data within the packet remains constant over a multi-packet communication interval.
Such information is hereinafter referred to as "static" information and generally remains unchanged for the duration of a conversation or session. While certain header information is altered during a packet's passage through a network, that information is constant for every packet occurring during a conversation. Such information is contained in the time-to-live (TTL) field, where it is decremented as a packet passes through nodes in a network. However, since a LANBRIDGE is stationary, succeeding packets in a conversation have identical TTL fields.
2~57~
In Fig. 4, static fields comprise header information including a destination address (6 bytes), source address (6 bytes), and packet type (2 bytes). In addition, within the IP header, the internet header length (IHL), type of service, flags, fragment offset, time to live (TTL), protocol, source address and destination address fields are also static. In the TCP header portion of the packet, static fields comprise the source port designation, destination port lo designation, data offset, flags, window, and urgent pointer.
As aforestated, all of the above data fields tend to remain lmçh~nged over a plurality of packets. While not shown in Fig. 4, it is understood by those skilled in the art that the TCP/IP packet format will generally be received as portion of a higher level packet format which will include further destination and source addresses as well as control data and protocol identification fields. Each of those can also be considered as static data.
A second group of fields within the TCP/IP packet format are termed "recalculatable". In essence, such fields carry information that can be derived from other fields and they are shown in Fig . 4 crossed hatched from upper right to lower left. They include the length, checksum, and frame check sequence (FCS) fields in the IP header.
There exists a standard for transparent bridges (802.1D) that stipulate the bridge must pass the FCS, end to end, ~ fied. The FCS is generated from all data bytes in the packet. On time-sensitive 2 ~ ri~ ~3 protocols, such as LAT, the extra time required to send all characters in the minimum length packet over low speed links, (e.g., 56 kilobits) may result in the packet being discarded due to a time-out. Thus, network managers may choose to override the 802.lD
requirement of passing the FCS end to end. This enables the tr~n! ission of a much shorter packets (e.g. elimination of FCS and many PAD bytes in the packet). This invention handles the FCS in a lo programmable manner. If the network manager selects FCS pass through, then the invention simply appends it as uncompressed data, at the end of a compressed packet. In the alternative, the FCS can be stripped from the packet and regenerated at the remote end by a receiving LANBRIDGE processor. In such a case, the FCS value is discarded.
As regards the length field (number of bytes in the packet), it may be zeroed under certain circumstances, with zeros written-in as placeholders. The length value is regenerated at the receiving end. The checksum value ~sum of values in IP header fields) is recalculated by subtracting it from a packet checksum value actually derived from examination of the packet header field values. The result is typically zero which compresses as a static field. At the receiving end, it is restored by the opposite calculation. By performing the subtraction, rather than just zeroing the checksum field, an error in the original checksum is duplicated at the remote end and lanbridge transparency is retained. A non-zero checksum may occur when the packet header has been computed, when an alternate checksum algorithm is experienced, etc.
- 2 ;~ 7 ~g The checksum field in the TCP header is treated similarly.
A third class of data, i.e., "Dynamic", contains fields that can change with every packet. Those fields are shown on Fig. 4, crosshatched from upper left to lower right and include in the IP header, the identification field, and in the TCP header, the sequence number and acknowl~g~ -nt number fields.
With respect to such fields, each is split into two parts, a high order part which is treated as semi-static and a low order part which is treated as dynamic. A dynamic field is one whose data changes with substantially every packet.
The final category of data in a packet is the user data. That data is compressed in accordance with previously known LZW techniques. However, the invention incorporates a number of important modifications, the first being that individual dictionaries corresponding to each protocol type are provided to enable improved compression. A second modification is that data ~pAn~ion is limited to at most, one bit over the entire length of the user-data portion of the packet.
As regards data expansion, this has occurred often in the prior art when the supposedly compressed data actually comes out as a longer data string than the non-compressed data. The invention assures that, for the heA~er~ at most the data Pxpansion is four bits and for the user data, 1 bit. As regards the header, only one twelve bit code is o~L~u~ for the compressed portion thereof. Thus, if only one byte i9 found in 14 2 ~ 7 ~
the dictionary, at most 4 additional bits will be transmitted If more than one byte is found, the invention provides true data compression.
As regards the user-data portion of a packet, if the encoder determines that data expansion will occur (i.e., the "compressed" output exceeds the input data stream length), it aborts the compression action. The user data is outputted as "clear" data and a bit is appended to indicate whether or not the data is encoded~ Even if, at the transmitting end the user data is transmitted as clear data, the transmitting ends dictionary has been updated in the course of the encoding action. Thus, at the receiving end, the decoder is required to proceed through the same updating action, even though non-compressed data was transmitted. This assures continued identity of the LZW dictionaries at both ends.
Thus, given the above packet data fields and their essential characteristics, the invention reorders them by segregating them as follows: static;
recalculatable: semi-static: and dynamic. The reordered fields occupy the same memory space as the original packet header. The static, recalculatable, and semi-static fields are then compressed using a modified LZW protocol with a dictionary table that is created specifically for the header data. Compression continues into the dynamic fields until the first incompressible field is encountered, at which point the remainder of the dynamic fields are not encoded, but are sent through as unencoded eight bit data.
Then, the type field of the LAN packet is used to seiect a "user-data" LZW dictionary, which dictionary 2 ~ 7 8 is used to compress the user-data portion of the LAN
packet. At the end of the packet, a bit is appended that indicates whether the user data field has been compressed. While it is preferred to use the LZW
algorithm, the compression of reformatted packet fields can be done using other string compression algorithms.
Turning now to Fig. 5, a reformatted TCP/IP packet is shown wherein the various categories of header fields have been moved within the packet to segregate static, semi-static and dynamic fields; and user data fields.
The recalculatable ~ields have been zeroed, and the sequence and acknowledgement number most significant 1~ words (MSW) segregated into the static region of the packet. The identification, sequence, and acknowledgement fields have been segregated into most and least significant portions, with the former being placed in the static region and the latter in the dynamic region.
once the packet is reformatted, as shown in Fig. 5, it iB ready to be compressed using an appropriate string compresslon algorithm. During compression, individual dictionary tables (see Fig. 6) will be employed. One is derived specifically for the header data, while the remaining are for user-data appearing in specific types of protocol packets. For instance, if a TCP
protocol is found, a separate TCP user-data dictionary will be created. Likewise, if ICMP, UDP, or LAT
protocol packets are detected, they too will have individual user data dictionaries created and will be utilized for succeeding protocol packets of an identical packet type. The segregation of static data 2~5~
into one area of the packet enables substantially improved compression to occur. Furthermore, the use of individual user-data dictionaries for sequences of identical packet types improves the probability that user data appearing in succeed;ng packets will be efficiently compressed.
Turning now to Figs. 7, 8, and 9, a flow diagram is shown which illustrates the algorithm employed within 0 LANR~Tnr~ processor 10 to accomplish both the reformatting and data compression functions.
Initially, LANBRIDGE 10 determines whether a packet has been received ~box 50) and if not, it continues waiting. If a packet has been received and placed into storage, the task of identifying the packet's type (box 52) is co~ -nced by eY~ ining the packet's type field. In accordance with the type indication found, a subroutine is called which has been designed precisely for the identified type. This subroutine is used to reformat the packet.
The reformatting operation commences by loading the indlvidual fields in the packet into registers within CPU 30 (box 54). Then, in accordance with the identified packet type, a subroutine serially writes back each of the fields, which is designated as StdtiC, into the packet. Then, the recalculatable fields are written out of their respective registers, recalculated, and written back into the packet, followed by semi-static and dynamic fields. As aforesaid, each of those fields is written into the packet in a serial fashion so that all static, recalculated and semi-static fields appear first in 2~5~7~
the packet, followed by the dynamic and user data fields (box 56).
While it is to be understood that the above-described method is the most preferred way for accomplishing the reformatting of the packet data structure, any other data processing technique which accomplishes the same result is acceptable. For instance, such reordering may be accomplished indirectly, and less efficiently, by not physically reformatting the packet, but by inserting pointers in a table to the various static regions in the packet and then accessing them in accordance with the table's notations.
Once the packet is reformatted, header data compression commences (see Fig. 8). For the purposes of this example, it is assumed that an LZW header dictionary table has been established and initialized with an initial character set of 256 entries. The algorithm co ~nces at the header's first reformatted byte (box 58) and matches the characters in the hea~r, character by character, with character strings stored in the header dictionary.
Assuming that a previous packet of the same type has been processed, it is likely that a complete or nearly complete match will occur with a long string in the dictionary table, and the matching characters will then be appropriately encoded using a 12 bit code.
Thus, as shown in box 60, if the total header is found tG match a fitring in the header dictionary table, the algorithm ouL~u~s a code for the matching characters and there is no LZW dictionary table update required.
If less than a total header match is found (box 62), a 18 2i)~78 12 bit code is outputted for the matching characters, and the remaining non-matched characters are transmitted unencoded (box 63) including the dynamic fields. In other words, no attempt at further compression is made once a non-match is found.
At this point, the algorithm departs from the LZW
te~.hn~ que which requires that a new table entry comprising the matching string plus the first character following the matching string be provided to the table. In this instance, the header dictionary table is updated by inserting, as a new entry, the entire static portion of the header (Box 62).
As is known to those skilled in the art, the LANBRIDGE
at the receiving end of the wide area network will receive the above-noted codes and will proceed to find the matching string within its own duplicate of the header dictionary table. In addition, it too will update its header table with the entire static header string, as did the transmitting end hANBRIDGE, so as to enable efficient decoding of encoded data.
once header data compression is completed, the user-data portion of the packet is data compressed (see Fig. 9). In this instance, however, a compression dictionary table is utilized which corresponds to the identified packet type (box 64) so as to assure a greater likelihood of string character match. Thus, using the LZW user-data dictionary corresponding to the packet type, the user-data is compressed using the standard LZW procedure. Codes are then Gu~pu~ted for the compressed data, followed 2a~ 7~
by the unencoded data (box 66) and the reformatted packet then loaded to an output buffer.
The packet's user-data bit length in the ou~L buffer is then ~- -red with the un-reformatted packet user-data bit length still in the input buffer (decision box 68). If the original packet length is shorter, the original input user-data bits are transmitted (box 70) followed by an indicator bit that compression has not occurred (box 71). Otherwise, the ~uL~u~ buffer user data bits are transmitted (box 72) and a bit is appended indicating whether the user-data field has been compressed (box 74).
Turning now to Figs. 10 - 1~, a further embodi -nt of the invention is shown for achieving additional data compression of dynamic fields. While a LAT packet is employed to explain this portion of the invention, it is to be understood that any packet having fields (e.g., sequence and/or acknowledgement identifiers) which are incremented or decremented as each packet is either received or transmitted also may employ this innovation.
Referring first to Fig. 10, a LAT packet is shown comprising 64 bytes of 8 bit data. The header's first 14 bytes include a destination address, a source address and a type field, and is the same for all Ethernet packets. The "type" field designates a LAT
packet in hexidecimal. The next 8 bytes (i.e. "LAT
type" through "MsgAck") constitute the remainder of the header.
.
~ ~3 Following the header are multiple bytes which include additional information regarding the packet and its data. In the packet shown in Fig. 10 only one user data byte is shown, the remaining bytes being taken up by "pad" bytes that are required to meet Ethernet/802.3 requirements for a in; frame length. The packet ends with a 32 bit frame check sequence (FCS).
The objective of this aspect of the invention is to increase the length of the data string that is likely to be found in the LZW dictionary. This is accomplished by realizing that (1) for packets used in a conversation, there are certain fields which do not change during the conversation and thus are static.
Those fields are arranged at the "top" of the packet.
(2) There are a number of sequence and acknowledgement fields (e.g. four in a LAT packet) which are in~L~ nted by a value of one during succeeding packet receptions/transmissions. By subtraction of the values in these fields between succeeding packets and substitution of the difference values (i.e. either zero or one) in the respective fields of the later packet, additional "delta" data compression is achieved. (3) The entire process is enabled by the creation of a connection table which includes a row, for each conversation, where updated values of the sequence and acknowledgement fields are stored. The operation of this aspect of the invention will be better understood from the following.
Once a LAT packet is identified, it is truncated (see Fig. 11) by eliminating all of the pad bytes and the frame check sequence bytes (assuming FCS pass through 2 ~ 5 ~
is disabled). Then, the remaining header portion is reconfigured, as shown in Fig. 12, so that bytes 0-15 include the following information: destination address, source address, type, destination ID, and source ID. This information does not change during a conversation. The next 4 bytes ~16 - 20) are dynamic and include the following data: message sequence, destination sequence, message acknowledgement and source sequence. These 4 bytes are either incremented by one or remain unchanged from packet to packet.
As shown in Figs. 13 and 14, a connection table is established and is allocated a row for each conversation in progress. For each conversation ~see box 100), the allocated row includes the following values from the last packet transmitted or received in the conversation: the message sequence number;
destination sequence number: message acknowledgement number: and source sequence number. Additionally, cross reference tables are established (box 102) which provide LZW dictionary-to-connection pointers and connection table-to-LZW dictionary pointers.
once a conversation is established and a first packet header has been received and entered into the LZW
dictionary, that header is assigned an LZW code value.
If a new hea~er is deteL ined to be a LAT header, its first 16 bytes are searched in the LZW dictionary (box 104) to determine if there is a match (box 106). If no match is found, indicating that this is a new conversation, the header is entered into the LZW
dictionary and the various tables are updated (box 108). If, on the other hand, the conversation is already in progress and a match is found, a cross 2 ~ 7 ~
reference table (using the LZW code value from the dictionary) is accessed to find where in the connection table the sequence and acknowledgement values from the previous packet's header is stored (box 110). Once those values are found, they are subtracted from each of the corresponding values in the new LAT header (box 112), and the remainders are substituted for the same fields in the original packet. In specific, since those fields are generally in~ -nted by 1 or left unchanged in succeeding packets, the resulting r ~ ~ i nder byte values, after subtraction, will either be all zeros or will exhibit only a one in the least significant bit position.
Since only 4 bytes are being considered (one per field), there will only be the possibility of 16 separate combinations of values in the bytes, after subtraction.
After the new remainder values have been appended to Z0 the static header string (box 114), the LZW dictionary is searched to determine if there is a match for the ext~n~ed string. The search begins at the LZW code assigned to the first 16 bytes and then continues to higher code values. If no match is found (box 118), the new string is entered and the tables are updated (box 120). If, a match is found, the associated LZW
code is substituted for the matching extended header string (box 122). At this point, the LZW code and remaining uncompressed header fields (if any) are transmitted (box 123). The procedure then increments to the user data compression procedure previously described (box 124).
2 ~ 7 ~
Generally, after one packet, there is a high probability of finding a match for the entire header in the dictionary. At most 16 frames in an entire conversation will not compress to the maximum amount, for the reason that they have not been seen before.
Compression is thus achieved for both the static header portions and certain dynamic portions that see an in~r,---nt after each packet is handled.
In summary, by reordering the information in the received packets into static and dynamic regions, longer, compressible character strings result and higher compression ratios are achieved using LZW and similar string compression algorithms. For instance, the hea~er dictionary does not contain the uncompressible dynamic fields, thereby conserving dictionary space for strings which will compress. In addition, by modifying the string compression algorithm to allocate a specific dictionary table to a packet's hea~er portion and immediately updating the header dictionary table to include an entire header character string, the learning time for the compression algorithm is substantially reduced. For instance, using the standard LZW algorithm, learning time for the hea~Rr will require 20 to 40 packets.
With this invention, the learning time encompasses one packet. Finally the use of individual user-data dictionary compression tables, by packet type or connection, enables higher efficiency compression.
It should be understood that the foregoing description is only illustrative of the invention. Various alternatives and modifications can be devised by those skilled in the art without departing from the 2 ~ 7 ~
invention. Accordingly, the present invention is intended to embrace all such alternatives, modifications and variances which fall within the scope of the appended claims.
Transactions on Information Theory, Vol. IT-24, No. 5, September, 1978, pp. 5306.
The Lempel-Ziv algorithm converts variable length strings of input symbols into fixed-length (or predictable length) codes. The symbol strings are selected so that all have almost equal probability of occurrence. Consequently, strings of frequently occurring symbols will contain more symbols than a string having infrequent symbols. This form of compression is effective at exploiting character frequency redundancy, character repetitions, and high usage pattern redundancy.
One of the first algorithms published by Lempel-Ziv (typically referenced to as LZ77) maintains a history buffer of the last N characters from the input data stream (typically 4,096) and encodes the output data stream as a sequence of symbols. If the character string is not found in the history buffer, it is encoded as a zero, followed by the unencoded eight bit character, resulting in a nine bit code. "Unencoded"
in this sense means the eight bit binary character which corresponds to the alpha-numeric character. If a character or stream of characters is found in the buffer, the stream is encoded as a 1, followed by an index and length in the dictionary. This thereby enables the receiving end to reconstruct, from its copy of the buffer, the transmitted data.
More recently, a modification to a subsequent Lempel-Ziv data compression technique (LZ 78) published by T. Welch, has become known as the LZW
algorithm. The LZW algorithm converts strings of varying lengths from an input data stream to fixed-length or predictable length codes, typically 12 bits in length. The premise of the algorithm is that frequently occurring strings contain more characters than infrequently occurring strings.
Initially an LZW dictionary or code table is empty, except for the first 256 character positions which contain basic alpha-numeric single character entries.
A new entry is created whenever a previously unseen string is encountered. The compressor searches the input stream to determine the longest match to a string stored in the dictionary. Each stored string comprises a prefix string and an extension chara¢ter.
Each string has a code signal associated with it. A
string is stored in the string table by, at least implicitly, storing the code signal for the string.
When a longest match between an input data character 7 ~
stream and a stored stream is determined, the code signal for the longest match is transmitted as the "compressed" code signal and a new string is stored in the string table. The prefix of the new string is the longest match of string characters and the suffix is an extension character which is the next data character from the input data that resulted in the longest match. Thus, as each compression occurs, the string lengths are increased by the addition of the extension character. Additional details of this algorithm can be found in U.S. Patent 4,558,302 to T.A. Welch, and in an article by Welch entitled "A
Technique For High-Perfol ~nce Data Compression" IEEE
Computer, June 1984, pp. 8-19.
While the LZW data compression algorithm is widely used, it does exhibit a number of shortcomings. For instance, during early stages in the construction of the dictionary, many data fragments (i.e. character lengths of 2, 3, or 4) will occupy large parts of the available dictionary storage. Thus, the amounts of compression available will be limited by the available strings. Often, in lieu of compressing the data, it will actually e~pAnd the data being transmitted. For instance, when only a single character is found to match, as will be the case in the early stages of dictionary construction, the outputting of a 12 bit code for an 8 bit input character will result in a 50%
increase in data.
Accordingly, it is an object of this invention to provide a data compression method particularly adapted for use with packetized data.
2a6~
It is another object of this invention to provide an improved data compression method which avoids the accumulation of short data strings in the compression dictionary.
s It is still another object of this invention to provide a data compression method that accommodates a plurality of protocols employing different type packets.
Yet another object of this invention is to provide an altered method of operation for the LZW compression algorithm that enables the attainment of improved compression results.
SUMMARY OF THE lNvrNl~lON
A data processing system is described that employs data packets which include at least static and dynamic fields, the static fields containing information that often remains constant during a multi-packet communication interval, and the dynamic fields containing information that may change for each packet. Many packets also include user-data fields.
A compression method is described which comprises:
reformatting each data packet by associating its static fields with a first packet region and its dynamic fields with a second packet region. The process then assembles a static table that includes static information from at least an initial data packet's first packet region. It then identifies static field information in a subsequent data packet's first packet region that is common to the information in the static table. Such common information is 2 ~
encoded so as to reduce its data length. The common static information is then replaced in the modified data packet with the encoded common static information and the modified data packet is then transmitted. A
similar action occurs with respect to user-data information. A single dictionary table is created for all packet headers, while separate dictionary tables are created for each user-data portion of a packet-type experienced in the communication network, thereby enabling better compression.
DESCRIPTION OF THE DRAWINGS
Fig. 1 is a block diagram of a LAN to LAN
communication network over a wide area network (WAN) link.
Fig. 2 is a software/hardware block diagram of the LANBRIDGE processor and modem portions of a WAN link.
Fig. 3 is a block diagram showing major hardware components of a LANBRIDGE processor.
Fig. 4 is a TCP/IP over Ethernet packet format.
Fig. 5 i~ the packet format of Fig. 4 which has been reformatted in accordance with the invention.
Fig. 6 is a list of compression tables employed to compress the data in the reformatted packet of Fig. 5.
Figs. 7-9 illustrate a high level block diagram describing the method of the invention.
- 2 ~
Fig. lO is a depiction of a LAT packet showing byte numbers plotted along the left side of the packet depiction and the data indicated within the packet bytes.
Fig. 11 is the depiction of Fig. 10 after the LAT
packet has been truncated.
Fig. 12 is a depiction of the truncated LAT packet of Fig. 11 after certain bytes have been "swapped" in their positions within the packet's header structure.
Figs. 13 and 14 illustrate a high level flow diagram describing a method for data compressing the swapped LAT packet header structure.
DE~ATT~n DESCRIPTION OF THE lNV~-~ ~ lON
Referring first to Fig. 1, a pair of LAN's A and B are shown, one located in Boston and one in Ottawa. Each LAN has att~ched thereto, various devices which are well known in the art. As aforestated, there is little or no need for data compression within each of LANs A or B. On the other hand, when data is to be transmitted from LAN A to LAN B, it will first pass through LANBRIDGE 10, where the packet data appearing on LAN A is au lessed in accordance with the invention. Such encoded data is then transmitted by modem 12 over WAN link 14 to modem 16. The received data is decv ~-essed by LANBRIDGE 18, and the packet structures appearing at the input to LANBRIDGE 10 are reconstructed and emplaced on LAN B.
2~3~Ps ~
In Fig. 2, a combined hardware/software block diagram of LANBRIDGE 10 is shown. Data packets appearing on LAN A are received by LAN interface devices 20 and passed into a random access memory (RAM) within 5 LANBRIDGE processor 10. Within LANBRIDGE processor 10 is software 22 which, in addition to performing routing and other functions, also performs reformatting and data compression functions.
A high level block diagram of LANBRIDGE processor 10 is shown in Fig. 3. A central processing unit 30 forms the heart of LANBRIDGE processor 10 and communicates with other elements of the system via bus 32. A LAN interface 20 is connected to bus 32, as is WAN interface 34 (provides a gateway for data to and from modem 12). An electrically alterable, programmable read only memory 36 and a RAM 38 provide storage functions for CPU 30. Within RAM 38 are a plurality of compression tables that are employed in the operation of the invention.
Returning to Fig. 2, bridge software 22 within LANBRIDGE processor 10 contains portions which first reformat each packet received from LAN A, and then function to compress data in the reformatted packet in accordance with data stored in the compression tables in RAM 38. The compressed data is then transmitted, through WAN interface 34 and onto the wide area network.
For the remainder of this portion of the description, it will be assumed that LANBRIDGE processor 10 has received a TCP/IP packet from LAN A. It will be further assumed that it is employing an Ethernet packet format. Such a format is shown in Fig. 4. In brief, the Transmission Control Protocol/Internet Protocol (TCP/IP) is a well known and widely used protocol. At a high level, it includes four protocols, a network access protocol (NAP), an internet protocol (IP), a transmission control protocol (TCP), and a process application protocol (PAP). Subsidiary to the internet protocol, there can be other protocols besides TCP, for instance, an internet control message protocol (ICMP), a user datagram protocol (UDP), etc. It is to be understood that this invention is equally applicable to other protocols whose packet formats contain the classes of data to be hereinafter described, (i.e. local area transport protocol (LAT)) and it is in no way limited to the TCP/IP format.
In ~ig. 4, the fields present in a TCP/IP packet are shown, with the number of bytes in each indicated to the left of the Fig. The information contained within each of the fields is shown within each field's boundary. Certain data within the packet remains constant over a multi-packet communication interval.
Such information is hereinafter referred to as "static" information and generally remains unchanged for the duration of a conversation or session. While certain header information is altered during a packet's passage through a network, that information is constant for every packet occurring during a conversation. Such information is contained in the time-to-live (TTL) field, where it is decremented as a packet passes through nodes in a network. However, since a LANBRIDGE is stationary, succeeding packets in a conversation have identical TTL fields.
2~57~
In Fig. 4, static fields comprise header information including a destination address (6 bytes), source address (6 bytes), and packet type (2 bytes). In addition, within the IP header, the internet header length (IHL), type of service, flags, fragment offset, time to live (TTL), protocol, source address and destination address fields are also static. In the TCP header portion of the packet, static fields comprise the source port designation, destination port lo designation, data offset, flags, window, and urgent pointer.
As aforestated, all of the above data fields tend to remain lmçh~nged over a plurality of packets. While not shown in Fig. 4, it is understood by those skilled in the art that the TCP/IP packet format will generally be received as portion of a higher level packet format which will include further destination and source addresses as well as control data and protocol identification fields. Each of those can also be considered as static data.
A second group of fields within the TCP/IP packet format are termed "recalculatable". In essence, such fields carry information that can be derived from other fields and they are shown in Fig . 4 crossed hatched from upper right to lower left. They include the length, checksum, and frame check sequence (FCS) fields in the IP header.
There exists a standard for transparent bridges (802.1D) that stipulate the bridge must pass the FCS, end to end, ~ fied. The FCS is generated from all data bytes in the packet. On time-sensitive 2 ~ ri~ ~3 protocols, such as LAT, the extra time required to send all characters in the minimum length packet over low speed links, (e.g., 56 kilobits) may result in the packet being discarded due to a time-out. Thus, network managers may choose to override the 802.lD
requirement of passing the FCS end to end. This enables the tr~n! ission of a much shorter packets (e.g. elimination of FCS and many PAD bytes in the packet). This invention handles the FCS in a lo programmable manner. If the network manager selects FCS pass through, then the invention simply appends it as uncompressed data, at the end of a compressed packet. In the alternative, the FCS can be stripped from the packet and regenerated at the remote end by a receiving LANBRIDGE processor. In such a case, the FCS value is discarded.
As regards the length field (number of bytes in the packet), it may be zeroed under certain circumstances, with zeros written-in as placeholders. The length value is regenerated at the receiving end. The checksum value ~sum of values in IP header fields) is recalculated by subtracting it from a packet checksum value actually derived from examination of the packet header field values. The result is typically zero which compresses as a static field. At the receiving end, it is restored by the opposite calculation. By performing the subtraction, rather than just zeroing the checksum field, an error in the original checksum is duplicated at the remote end and lanbridge transparency is retained. A non-zero checksum may occur when the packet header has been computed, when an alternate checksum algorithm is experienced, etc.
- 2 ;~ 7 ~g The checksum field in the TCP header is treated similarly.
A third class of data, i.e., "Dynamic", contains fields that can change with every packet. Those fields are shown on Fig. 4, crosshatched from upper left to lower right and include in the IP header, the identification field, and in the TCP header, the sequence number and acknowl~g~ -nt number fields.
With respect to such fields, each is split into two parts, a high order part which is treated as semi-static and a low order part which is treated as dynamic. A dynamic field is one whose data changes with substantially every packet.
The final category of data in a packet is the user data. That data is compressed in accordance with previously known LZW techniques. However, the invention incorporates a number of important modifications, the first being that individual dictionaries corresponding to each protocol type are provided to enable improved compression. A second modification is that data ~pAn~ion is limited to at most, one bit over the entire length of the user-data portion of the packet.
As regards data expansion, this has occurred often in the prior art when the supposedly compressed data actually comes out as a longer data string than the non-compressed data. The invention assures that, for the heA~er~ at most the data Pxpansion is four bits and for the user data, 1 bit. As regards the header, only one twelve bit code is o~L~u~ for the compressed portion thereof. Thus, if only one byte i9 found in 14 2 ~ 7 ~
the dictionary, at most 4 additional bits will be transmitted If more than one byte is found, the invention provides true data compression.
As regards the user-data portion of a packet, if the encoder determines that data expansion will occur (i.e., the "compressed" output exceeds the input data stream length), it aborts the compression action. The user data is outputted as "clear" data and a bit is appended to indicate whether or not the data is encoded~ Even if, at the transmitting end the user data is transmitted as clear data, the transmitting ends dictionary has been updated in the course of the encoding action. Thus, at the receiving end, the decoder is required to proceed through the same updating action, even though non-compressed data was transmitted. This assures continued identity of the LZW dictionaries at both ends.
Thus, given the above packet data fields and their essential characteristics, the invention reorders them by segregating them as follows: static;
recalculatable: semi-static: and dynamic. The reordered fields occupy the same memory space as the original packet header. The static, recalculatable, and semi-static fields are then compressed using a modified LZW protocol with a dictionary table that is created specifically for the header data. Compression continues into the dynamic fields until the first incompressible field is encountered, at which point the remainder of the dynamic fields are not encoded, but are sent through as unencoded eight bit data.
Then, the type field of the LAN packet is used to seiect a "user-data" LZW dictionary, which dictionary 2 ~ 7 8 is used to compress the user-data portion of the LAN
packet. At the end of the packet, a bit is appended that indicates whether the user data field has been compressed. While it is preferred to use the LZW
algorithm, the compression of reformatted packet fields can be done using other string compression algorithms.
Turning now to Fig. 5, a reformatted TCP/IP packet is shown wherein the various categories of header fields have been moved within the packet to segregate static, semi-static and dynamic fields; and user data fields.
The recalculatable ~ields have been zeroed, and the sequence and acknowledgement number most significant 1~ words (MSW) segregated into the static region of the packet. The identification, sequence, and acknowledgement fields have been segregated into most and least significant portions, with the former being placed in the static region and the latter in the dynamic region.
once the packet is reformatted, as shown in Fig. 5, it iB ready to be compressed using an appropriate string compresslon algorithm. During compression, individual dictionary tables (see Fig. 6) will be employed. One is derived specifically for the header data, while the remaining are for user-data appearing in specific types of protocol packets. For instance, if a TCP
protocol is found, a separate TCP user-data dictionary will be created. Likewise, if ICMP, UDP, or LAT
protocol packets are detected, they too will have individual user data dictionaries created and will be utilized for succeeding protocol packets of an identical packet type. The segregation of static data 2~5~
into one area of the packet enables substantially improved compression to occur. Furthermore, the use of individual user-data dictionaries for sequences of identical packet types improves the probability that user data appearing in succeed;ng packets will be efficiently compressed.
Turning now to Figs. 7, 8, and 9, a flow diagram is shown which illustrates the algorithm employed within 0 LANR~Tnr~ processor 10 to accomplish both the reformatting and data compression functions.
Initially, LANBRIDGE 10 determines whether a packet has been received ~box 50) and if not, it continues waiting. If a packet has been received and placed into storage, the task of identifying the packet's type (box 52) is co~ -nced by eY~ ining the packet's type field. In accordance with the type indication found, a subroutine is called which has been designed precisely for the identified type. This subroutine is used to reformat the packet.
The reformatting operation commences by loading the indlvidual fields in the packet into registers within CPU 30 (box 54). Then, in accordance with the identified packet type, a subroutine serially writes back each of the fields, which is designated as StdtiC, into the packet. Then, the recalculatable fields are written out of their respective registers, recalculated, and written back into the packet, followed by semi-static and dynamic fields. As aforesaid, each of those fields is written into the packet in a serial fashion so that all static, recalculated and semi-static fields appear first in 2~5~7~
the packet, followed by the dynamic and user data fields (box 56).
While it is to be understood that the above-described method is the most preferred way for accomplishing the reformatting of the packet data structure, any other data processing technique which accomplishes the same result is acceptable. For instance, such reordering may be accomplished indirectly, and less efficiently, by not physically reformatting the packet, but by inserting pointers in a table to the various static regions in the packet and then accessing them in accordance with the table's notations.
Once the packet is reformatted, header data compression commences (see Fig. 8). For the purposes of this example, it is assumed that an LZW header dictionary table has been established and initialized with an initial character set of 256 entries. The algorithm co ~nces at the header's first reformatted byte (box 58) and matches the characters in the hea~r, character by character, with character strings stored in the header dictionary.
Assuming that a previous packet of the same type has been processed, it is likely that a complete or nearly complete match will occur with a long string in the dictionary table, and the matching characters will then be appropriately encoded using a 12 bit code.
Thus, as shown in box 60, if the total header is found tG match a fitring in the header dictionary table, the algorithm ouL~u~s a code for the matching characters and there is no LZW dictionary table update required.
If less than a total header match is found (box 62), a 18 2i)~78 12 bit code is outputted for the matching characters, and the remaining non-matched characters are transmitted unencoded (box 63) including the dynamic fields. In other words, no attempt at further compression is made once a non-match is found.
At this point, the algorithm departs from the LZW
te~.hn~ que which requires that a new table entry comprising the matching string plus the first character following the matching string be provided to the table. In this instance, the header dictionary table is updated by inserting, as a new entry, the entire static portion of the header (Box 62).
As is known to those skilled in the art, the LANBRIDGE
at the receiving end of the wide area network will receive the above-noted codes and will proceed to find the matching string within its own duplicate of the header dictionary table. In addition, it too will update its header table with the entire static header string, as did the transmitting end hANBRIDGE, so as to enable efficient decoding of encoded data.
once header data compression is completed, the user-data portion of the packet is data compressed (see Fig. 9). In this instance, however, a compression dictionary table is utilized which corresponds to the identified packet type (box 64) so as to assure a greater likelihood of string character match. Thus, using the LZW user-data dictionary corresponding to the packet type, the user-data is compressed using the standard LZW procedure. Codes are then Gu~pu~ted for the compressed data, followed 2a~ 7~
by the unencoded data (box 66) and the reformatted packet then loaded to an output buffer.
The packet's user-data bit length in the ou~L buffer is then ~- -red with the un-reformatted packet user-data bit length still in the input buffer (decision box 68). If the original packet length is shorter, the original input user-data bits are transmitted (box 70) followed by an indicator bit that compression has not occurred (box 71). Otherwise, the ~uL~u~ buffer user data bits are transmitted (box 72) and a bit is appended indicating whether the user-data field has been compressed (box 74).
Turning now to Figs. 10 - 1~, a further embodi -nt of the invention is shown for achieving additional data compression of dynamic fields. While a LAT packet is employed to explain this portion of the invention, it is to be understood that any packet having fields (e.g., sequence and/or acknowledgement identifiers) which are incremented or decremented as each packet is either received or transmitted also may employ this innovation.
Referring first to Fig. 10, a LAT packet is shown comprising 64 bytes of 8 bit data. The header's first 14 bytes include a destination address, a source address and a type field, and is the same for all Ethernet packets. The "type" field designates a LAT
packet in hexidecimal. The next 8 bytes (i.e. "LAT
type" through "MsgAck") constitute the remainder of the header.
.
~ ~3 Following the header are multiple bytes which include additional information regarding the packet and its data. In the packet shown in Fig. 10 only one user data byte is shown, the remaining bytes being taken up by "pad" bytes that are required to meet Ethernet/802.3 requirements for a in; frame length. The packet ends with a 32 bit frame check sequence (FCS).
The objective of this aspect of the invention is to increase the length of the data string that is likely to be found in the LZW dictionary. This is accomplished by realizing that (1) for packets used in a conversation, there are certain fields which do not change during the conversation and thus are static.
Those fields are arranged at the "top" of the packet.
(2) There are a number of sequence and acknowledgement fields (e.g. four in a LAT packet) which are in~L~ nted by a value of one during succeeding packet receptions/transmissions. By subtraction of the values in these fields between succeeding packets and substitution of the difference values (i.e. either zero or one) in the respective fields of the later packet, additional "delta" data compression is achieved. (3) The entire process is enabled by the creation of a connection table which includes a row, for each conversation, where updated values of the sequence and acknowledgement fields are stored. The operation of this aspect of the invention will be better understood from the following.
Once a LAT packet is identified, it is truncated (see Fig. 11) by eliminating all of the pad bytes and the frame check sequence bytes (assuming FCS pass through 2 ~ 5 ~
is disabled). Then, the remaining header portion is reconfigured, as shown in Fig. 12, so that bytes 0-15 include the following information: destination address, source address, type, destination ID, and source ID. This information does not change during a conversation. The next 4 bytes ~16 - 20) are dynamic and include the following data: message sequence, destination sequence, message acknowledgement and source sequence. These 4 bytes are either incremented by one or remain unchanged from packet to packet.
As shown in Figs. 13 and 14, a connection table is established and is allocated a row for each conversation in progress. For each conversation ~see box 100), the allocated row includes the following values from the last packet transmitted or received in the conversation: the message sequence number;
destination sequence number: message acknowledgement number: and source sequence number. Additionally, cross reference tables are established (box 102) which provide LZW dictionary-to-connection pointers and connection table-to-LZW dictionary pointers.
once a conversation is established and a first packet header has been received and entered into the LZW
dictionary, that header is assigned an LZW code value.
If a new hea~er is deteL ined to be a LAT header, its first 16 bytes are searched in the LZW dictionary (box 104) to determine if there is a match (box 106). If no match is found, indicating that this is a new conversation, the header is entered into the LZW
dictionary and the various tables are updated (box 108). If, on the other hand, the conversation is already in progress and a match is found, a cross 2 ~ 7 ~
reference table (using the LZW code value from the dictionary) is accessed to find where in the connection table the sequence and acknowledgement values from the previous packet's header is stored (box 110). Once those values are found, they are subtracted from each of the corresponding values in the new LAT header (box 112), and the remainders are substituted for the same fields in the original packet. In specific, since those fields are generally in~ -nted by 1 or left unchanged in succeeding packets, the resulting r ~ ~ i nder byte values, after subtraction, will either be all zeros or will exhibit only a one in the least significant bit position.
Since only 4 bytes are being considered (one per field), there will only be the possibility of 16 separate combinations of values in the bytes, after subtraction.
After the new remainder values have been appended to Z0 the static header string (box 114), the LZW dictionary is searched to determine if there is a match for the ext~n~ed string. The search begins at the LZW code assigned to the first 16 bytes and then continues to higher code values. If no match is found (box 118), the new string is entered and the tables are updated (box 120). If, a match is found, the associated LZW
code is substituted for the matching extended header string (box 122). At this point, the LZW code and remaining uncompressed header fields (if any) are transmitted (box 123). The procedure then increments to the user data compression procedure previously described (box 124).
2 ~ 7 ~
Generally, after one packet, there is a high probability of finding a match for the entire header in the dictionary. At most 16 frames in an entire conversation will not compress to the maximum amount, for the reason that they have not been seen before.
Compression is thus achieved for both the static header portions and certain dynamic portions that see an in~r,---nt after each packet is handled.
In summary, by reordering the information in the received packets into static and dynamic regions, longer, compressible character strings result and higher compression ratios are achieved using LZW and similar string compression algorithms. For instance, the hea~er dictionary does not contain the uncompressible dynamic fields, thereby conserving dictionary space for strings which will compress. In addition, by modifying the string compression algorithm to allocate a specific dictionary table to a packet's hea~er portion and immediately updating the header dictionary table to include an entire header character string, the learning time for the compression algorithm is substantially reduced. For instance, using the standard LZW algorithm, learning time for the hea~Rr will require 20 to 40 packets.
With this invention, the learning time encompasses one packet. Finally the use of individual user-data dictionary compression tables, by packet type or connection, enables higher efficiency compression.
It should be understood that the foregoing description is only illustrative of the invention. Various alternatives and modifications can be devised by those skilled in the art without departing from the 2 ~ 7 ~
invention. Accordingly, the present invention is intended to embrace all such alternatives, modifications and variances which fall within the scope of the appended claims.
Claims (14)
1. In a data processing system that employs packets, a packet including at least static and dynamic fields, said static fields including information which often remains constant during a multi-packet communication interval, said dynamic fields including information which may change for each packet, some said packets also including user-data fields, a method for compressing the contents of said data packets for transmission from one to another portion of the system, the method comprising:
a. reformatting each packet by associating its static fields with a first packet region, and its dynamic-fields with second packet region;
b. assembling a static table that includes static field information from at least an initial data packet's first packet region and assigning a code value thereto;
c. identifying static field information associated with a subsequent packet's first packet region that is common to information in said static table, and selecting said code value assigned to said common information;
d. modifying said subsequent packet by replacing said common, static field information with said selected code value; and e. transmitting said modified subsequent data packet.
a. reformatting each packet by associating its static fields with a first packet region, and its dynamic-fields with second packet region;
b. assembling a static table that includes static field information from at least an initial data packet's first packet region and assigning a code value thereto;
c. identifying static field information associated with a subsequent packet's first packet region that is common to information in said static table, and selecting said code value assigned to said common information;
d. modifying said subsequent packet by replacing said common, static field information with said selected code value; and e. transmitting said modified subsequent data packet.
2. The method of claim 1, wherein said reformatting step occurs with respect to data contained within header fields appearing in each packet and the method commences with the step of segregating header fields and user-data fields.
3. The method of claim 2 wherein said reformatting step further comprises rearranging each packet to place its static fields into said first packet region and its dynamic fields into said second packet region.
4. The method as recited in claim 1, comprising the additional steps of:
f. assembling a user-data table that includes user-data field information from at least an initial packet and assigning a code value thereto;
g. identifying user-data field information in subsequent packets that is common to user-data field information in said user-data table and selecting said code value assigned to said common information;
h. modifying said subsequent packet by replacing said common user-data field information with said selected code value.
f. assembling a user-data table that includes user-data field information from at least an initial packet and assigning a code value thereto;
g. identifying user-data field information in subsequent packets that is common to user-data field information in said user-data table and selecting said code value assigned to said common information;
h. modifying said subsequent packet by replacing said common user-data field information with said selected code value.
5. The method of claim 4, wherein said data processing system employs a plurality of protocols, each protocol employing a unique packet structure, said method comprising the added step of:
i. establishing for each one of a group of selected protocols, a user-data table, whereby user-data in a packet belonging to a said selected protocol, is compressed in relation to common user-data in a said user-data table specific for said protocol.
i. establishing for each one of a group of selected protocols, a user-data table, whereby user-data in a packet belonging to a said selected protocol, is compressed in relation to common user-data in a said user-data table specific for said protocol.
6. The method as recited in claim 5 further comprising the step of:
d1. appending a coded symbol to a said packet which indicates whether it has been data compressed or not.
d1. appending a coded symbol to a said packet which indicates whether it has been data compressed or not.
7. The method of claim 4, comprising the additional step of:
i. determining if said selected user data table code value, in addition to remaining uncompressed user-data of a packet, is greater in bit length than all uncompressed user-data in a said packet, and if so, modifying step e by transmitting said uncompressed user-data from said packet in lieu of said user data table code value and remaining uncompressed user-data.
i. determining if said selected user data table code value, in addition to remaining uncompressed user-data of a packet, is greater in bit length than all uncompressed user-data in a said packet, and if so, modifying step e by transmitting said uncompressed user-data from said packet in lieu of said user data table code value and remaining uncompressed user-data.
8. A method for compressing packets in a data communication system, said packets comprising header fields that include at least static and dynamic data, said method comprising:
a. restructuring each said header field to segregate static data field strings from dynamic data field strings;
b. assembling a compression table that includes restructured static data field strings from packet headers, and a code value assigned to each said string;
c. examining a restructured static data field string from a new packet header to determine if a matching static data field string is included in said compression table, and if a match is found, selecting the code value assigned to said matching static data field string in said table; and d. substituting said assigned code value for said matching restructured static data field string when said packet is transmitted in said data communication system.
a. restructuring each said header field to segregate static data field strings from dynamic data field strings;
b. assembling a compression table that includes restructured static data field strings from packet headers, and a code value assigned to each said string;
c. examining a restructured static data field string from a new packet header to determine if a matching static data field string is included in said compression table, and if a match is found, selecting the code value assigned to said matching static data field string in said table; and d. substituting said assigned code value for said matching restructured static data field string when said packet is transmitted in said data communication system.
9. The method of claim 8, further comprising the added steps of:
e. assembling a connection table for conversations over said system, by entering therein selected dynamic data field strings from packets in said conversations;
f. for a subsequent packet in a said conversation, computing differences between said subsequent data packet's selected header dynamic data field strings and said selected dynamic data packet strings for said conversation in said connection table;
g. entering in said compression table said static header field data string from said packets in said conversation and said computed differences and assigning a code value thereto;
h. for subsequent packets, repeating step f, and updating said connection table each time, with a said subsequent packet's selected dynamic data field string; and i. for each subsequent packet, searching said compression table to find a match for a said subsequent packet's static header data field string and said computed differences, and when a match is found, employing said assigned code value in lieu thereof, as recited in step (d).
e. assembling a connection table for conversations over said system, by entering therein selected dynamic data field strings from packets in said conversations;
f. for a subsequent packet in a said conversation, computing differences between said subsequent data packet's selected header dynamic data field strings and said selected dynamic data packet strings for said conversation in said connection table;
g. entering in said compression table said static header field data string from said packets in said conversation and said computed differences and assigning a code value thereto;
h. for subsequent packets, repeating step f, and updating said connection table each time, with a said subsequent packet's selected dynamic data field string; and i. for each subsequent packet, searching said compression table to find a match for a said subsequent packet's static header data field string and said computed differences, and when a match is found, employing said assigned code value in lieu thereof, as recited in step (d).
10. The method of claim 9 wherein said selected header dynamic data string includes data that may be incremented by an integer in succeeding packets during a conversation.
11. The method of claim 8, wherein each said packet header contains a semistatic field, a first portion thereof changing with every packet and a second portion thereof remaining static over a plurality of packets, said method comprising the added steps of:
al. segregating said second portion of each packet header with said static data field string of said header field of a said packet.
al. segregating said second portion of each packet header with said static data field string of said header field of a said packet.
12. The method of claim 8 wherein said data communications system accommodates a plurality of protocols each protocol employing a unique packet structure, wherein said method comprises the added steps of:
e. assembling for each packet structure, a user data table and entering therein user-data strings from said unique packet structures and assigning a code value thereto; and f. determining if a user data string from a subsequent packet structure matches a user data string in said user data table and if a match is found, substituting said assigned code value for said matching user data string.
e. assembling for each packet structure, a user data table and entering therein user-data strings from said unique packet structures and assigning a code value thereto; and f. determining if a user data string from a subsequent packet structure matches a user data string in said user data table and if a match is found, substituting said assigned code value for said matching user data string.
13. The method of claim 12, including the further step of:
g. inhibiting the substitution of step f if said assigned code value and remaining non-matching user data exhibits a data length longer than all user-data in a said packet.
g. inhibiting the substitution of step f if said assigned code value and remaining non-matching user data exhibits a data length longer than all user-data in a said packet.
14. The method of claim 8, wherein said packet contains a checksum field, said method including the further step of:
c1. calculating a new checksum for said packet, subtracting said new checksum value from the value in said checksum field to obtain a difference value and replacing the value in said checksum field with said difference value.
c1. calculating a new checksum for said packet, subtracting said new checksum value from the value in said checksum field to obtain a difference value and replacing the value in said checksum field with said difference value.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US68935991A | 1991-04-22 | 1991-04-22 | |
| US07/689,359 | 1991-04-22 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CA2065578A1 CA2065578A1 (en) | 1992-10-23 |
| CA2065578C true CA2065578C (en) | 1999-02-23 |
Family
ID=24768105
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA002065578A Expired - Lifetime CA2065578C (en) | 1991-04-22 | 1992-04-08 | Packet-based data compression method |
Country Status (2)
| Country | Link |
|---|---|
| US (1) | US5293379A (en) |
| CA (1) | CA2065578C (en) |
Families Citing this family (174)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA2077271C (en) * | 1991-12-13 | 1998-07-28 | David J. Craft | Method and apparatus for compressing data |
| FR2687259B1 (en) * | 1992-02-11 | 1994-05-06 | Ouest Standard Telematique Sa | DATA COMPRESSION METHOD FOR PROTOCOL DATA UNIT TRANSMISSION SYSTEM, DECOMPRESSION METHOD, AND CORRESPONDING DEVICE. |
| JP3630441B2 (en) * | 1992-12-21 | 2005-03-16 | ソニー株式会社 | Transmission method, reception method, communication method, bidirectional bus system, and electronic device |
| JP3314438B2 (en) * | 1993-02-22 | 2002-08-12 | 株式会社日立製作所 | Data communication device |
| EP0688477A1 (en) * | 1993-03-10 | 1995-12-27 | Awacs Communications (Nz) Limited | Enhanced one way radio seven bit data network |
| JPH0778115A (en) * | 1993-09-08 | 1995-03-20 | Fujitsu Ltd | Network definition changing system |
| JPH0795230A (en) * | 1993-09-20 | 1995-04-07 | Fujitsu Ltd | Processing system for subscriber's information in connectionless data service |
| US5495484A (en) * | 1993-10-12 | 1996-02-27 | Dsc Communications Corporation | Distributed telecommunications switching system |
| SE515361C2 (en) * | 1993-12-22 | 2001-07-23 | Eritel Ab | Transmission of spoken messages |
| US5563595A (en) * | 1993-12-23 | 1996-10-08 | International Business Machines Corporation | Method and apparatus for compressing data |
| US5414701A (en) * | 1994-07-22 | 1995-05-09 | Motorola, Inc. | Method and data structure for performing address compression in an asynchronous transfer mode (ATM) system |
| US5535199A (en) * | 1994-09-06 | 1996-07-09 | Sun Microsystems, Inc. | TCP/IP header compression X.25 networks |
| CA2143374A1 (en) * | 1994-09-21 | 1996-03-22 | Isaac Shpantzer | Data communication system with adaptive concatenation |
| US5550984A (en) * | 1994-12-07 | 1996-08-27 | Matsushita Electric Corporation Of America | Security system for preventing unauthorized communications between networks by translating communications received in ip protocol to non-ip protocol to remove address and routing services information |
| US5841769A (en) * | 1994-12-16 | 1998-11-24 | Nec Corporation | Data message routing system for mobile or fixed terminals within a network |
| US5659684A (en) * | 1995-02-03 | 1997-08-19 | Isdn Systems Corporation | Methods and apparatus for interconnecting personal computers (PCs) and local area networks (LANs) using packet protocols transmitted over a digital data service (DDS) |
| US5592486A (en) * | 1995-03-17 | 1997-01-07 | Advanced Micro Devices, Inc. | System and method for efficiently monitoring information in a network having a plurality of repeaters |
| EP0761041A1 (en) * | 1995-03-24 | 1997-03-12 | Motorola, Inc. | Data compression encoder/decoder and method for efficient duplicate string handling |
| US5581558A (en) * | 1995-03-29 | 1996-12-03 | Lucent Technologies Inc. | Apparatus for bridging non-compatible network architectures |
| KR100224815B1 (en) * | 1995-06-23 | 1999-10-15 | 윤종용 | Data compression and expansion method |
| US5657452A (en) * | 1995-09-08 | 1997-08-12 | U.S. Robotics Corp. | Transparent support of protocol and data compression features for data communication |
| US5724355A (en) * | 1995-10-24 | 1998-03-03 | At&T Corp | Network access to internet and stored multimedia services from a terminal supporting the H.320 protocol |
| US5841764A (en) * | 1995-10-30 | 1998-11-24 | Ericsson Inc. | Method and apparatus for permitting a radio to originate and receive data messages in a data communications network |
| US5850526A (en) * | 1996-02-07 | 1998-12-15 | Kingston Technology Co. | LAN station for determining the destination LAN station is capable of decompressing by comparing destination address to block of addresses assigned by a LAN manufacturer |
| US5867114A (en) * | 1996-02-29 | 1999-02-02 | Mitel Corporation | Method and apparatus for performing data compression |
| US6115384A (en) * | 1996-06-20 | 2000-09-05 | Fourelle Systems, Inc | Gateway architecture for data communication bandwidth-constrained and charge-by-use networks |
| US6094453A (en) * | 1996-10-11 | 2000-07-25 | Digital Accelerator Corporation | Digital data compression with quad-tree coding of header file |
| JPH10173659A (en) * | 1996-12-05 | 1998-06-26 | Uniden Corp | Transmission device, reception device, packet transmission system and packet signal transmission method |
| US5987022A (en) * | 1996-12-27 | 1999-11-16 | Motorola, Inc. | Method for transmitting multiple-protocol packetized data |
| DE69832593T2 (en) * | 1997-03-07 | 2006-08-17 | Intelligent Compression Technologies, Falmouth | NETWORK FOR DATA CODING |
| US5832235A (en) * | 1997-03-26 | 1998-11-03 | Hewlett-Packard Co. | System and method for pattern matching using checksums |
| US5983223A (en) * | 1997-05-06 | 1999-11-09 | Novell, Inc. | Method and apparatus for determining a longest matching prefix from a dictionary of prefixes |
| JP3523019B2 (en) * | 1997-06-18 | 2004-04-26 | 富士通株式会社 | Frame relay device |
| US6075798A (en) * | 1997-06-20 | 2000-06-13 | Lucent Technologies Inc. | Extended header for use in ATM adaptation layer type 2 packets |
| US6052738A (en) * | 1997-06-30 | 2000-04-18 | Sun Microsystems, Inc. | Method and apparatus in a packet routing switch for controlling access at different data rates to a shared memory |
| US6119196A (en) * | 1997-06-30 | 2000-09-12 | Sun Microsystems, Inc. | System having multiple arbitrating levels for arbitrating access to a shared memory by network ports operating at different data rates |
| US6128666A (en) * | 1997-06-30 | 2000-10-03 | Sun Microsystems, Inc. | Distributed VLAN mechanism for packet field replacement in a multi-layered switched network element using a control field/signal for indicating modification of a packet with a database search engine |
| US6016310A (en) * | 1997-06-30 | 2000-01-18 | Sun Microsystems, Inc. | Trunking support in a high performance network device |
| US5938736A (en) * | 1997-06-30 | 1999-08-17 | Sun Microsystems, Inc. | Search engine architecture for a high performance multi-layer switch element |
| US6246680B1 (en) | 1997-06-30 | 2001-06-12 | Sun Microsystems, Inc. | Highly integrated multi-layer switch element architecture |
| US6044418A (en) * | 1997-06-30 | 2000-03-28 | Sun Microsystems, Inc. | Method and apparatus for dynamically resizing queues utilizing programmable partition pointers |
| US6014380A (en) * | 1997-06-30 | 2000-01-11 | Sun Microsystems, Inc. | Mechanism for packet field replacement in a multi-layer distributed network element |
| US6081522A (en) * | 1997-06-30 | 2000-06-27 | Sun Microsystems, Inc. | System and method for a multi-layer network element |
| US6049528A (en) * | 1997-06-30 | 2000-04-11 | Sun Microsystems, Inc. | Trunking ethernet-compatible networks |
| US6044087A (en) * | 1997-06-30 | 2000-03-28 | Sun Microsystems, Inc. | Interface for a highly integrated ethernet network element |
| US5920566A (en) * | 1997-06-30 | 1999-07-06 | Sun Microsystems, Inc. | Routing in a multi-layer distributed network element |
| US6094435A (en) * | 1997-06-30 | 2000-07-25 | Sun Microsystems, Inc. | System and method for a quality of service in a multi-layer network element |
| US6118760A (en) * | 1997-06-30 | 2000-09-12 | Sun Microsystems, Inc. | Management of entries in a network element forwarding memory |
| US6088356A (en) * | 1997-06-30 | 2000-07-11 | Sun Microsystems, Inc. | System and method for a multi-layer network element |
| US6081512A (en) * | 1997-06-30 | 2000-06-27 | Sun Microsystems, Inc. | Spanning tree support in a high performance network device |
| US6529512B1 (en) * | 1997-07-15 | 2003-03-04 | International Business Machines Corporation | Statistical method of data compression and decompression |
| US6839330B1 (en) * | 1997-07-15 | 2005-01-04 | Viasat, Inc. | Method and apparatus for improving asynchronous transfer mode operation over noisy, high speed wireless links |
| US6229823B1 (en) * | 1997-08-01 | 2001-05-08 | Paradyne Corporation | System and method for the compression of proprietary encapsulations |
| US6041054A (en) * | 1997-09-24 | 2000-03-21 | Telefonaktiebolaget Lm Ericsson | Efficient transport of internet protocol packets using asynchronous transfer mode adaptation layer two |
| US6032197A (en) * | 1997-09-25 | 2000-02-29 | Microsoft Corporation | Data packet header compression for unidirectional transmission |
| US6026093A (en) * | 1997-10-02 | 2000-02-15 | Sun Microsystems, Inc. | Mechanism for dispatching data units via a telecommunications network |
| US6473425B1 (en) | 1997-10-02 | 2002-10-29 | Sun Microsystems, Inc. | Mechanism for dispatching packets via a telecommunications network |
| US6151602A (en) * | 1997-11-07 | 2000-11-21 | Inprise Corporation | Database system with methods providing a platform-independent self-describing data packet for transmitting information |
| US5966373A (en) * | 1997-12-10 | 1999-10-12 | L-3 Communications Corporation | Waveform and frame structure for a fixed wireless loop synchronous CDMA communications system |
| US6249530B1 (en) | 1997-12-22 | 2001-06-19 | Sun Microsystems, Inc. | Network bandwidth control |
| US6115394A (en) * | 1998-03-04 | 2000-09-05 | Ericsson Inc. | Methods, apparatus and computer program products for packet transport over wireless communication links |
| US6920143B1 (en) * | 1998-03-12 | 2005-07-19 | Brooktrout, Inc. | Computer telephony system using multiple hardware platforms to provide telephony services |
| AU4389099A (en) * | 1998-06-23 | 2000-01-10 | Infit Communications Ltd. | Content storage and redundancy elimination |
| WO2000010305A1 (en) * | 1998-08-14 | 2000-02-24 | Citrix Systems, Inc. | Apparatus and method for extracting algorithmic information from a message stream |
| US6624761B2 (en) | 1998-12-11 | 2003-09-23 | Realtime Data, Llc | Content independent data compression method and system |
| AU3123800A (en) * | 1998-12-22 | 2000-07-12 | Expand Networks Ltd. | Synthetic history for adaptive data compression |
| US6314095B1 (en) * | 1999-02-11 | 2001-11-06 | Motorola, Inc. | Method and apparatus for a high-speed multimedia content switch with compressed internet protocol header |
| FI107000B (en) | 1999-02-17 | 2001-05-15 | Nokia Mobile Phones Ltd | Title compression in real-time services |
| US6396420B1 (en) * | 1999-02-26 | 2002-05-28 | Autodesk, Inc. | Delta compression of floating point coordinate data |
| US6604158B1 (en) * | 1999-03-11 | 2003-08-05 | Realtime Data, Llc | System and methods for accelerated data storage and retrieval |
| US6601104B1 (en) * | 1999-03-11 | 2003-07-29 | Realtime Data Llc | System and methods for accelerated data storage and retrieval |
| US20020091636A1 (en) * | 1999-03-25 | 2002-07-11 | Nortel Networks Corporation | Capturing quality of service |
| US7167860B1 (en) | 1999-03-25 | 2007-01-23 | Nortel Networks Limited | Fault tolerance for network accounting architecture |
| US7243143B1 (en) * | 1999-03-25 | 2007-07-10 | Nortel Networks Limited | Flow probe connectivity determination |
| US6751663B1 (en) | 1999-03-25 | 2004-06-15 | Nortel Networks Limited | System wide flow aggregation process for aggregating network activity records |
| US6594276B1 (en) | 1999-04-01 | 2003-07-15 | Nokia Corporation | Apparatus and associated method for communicating multimedia information upon a communication link |
| SE521700C2 (en) * | 1999-05-21 | 2003-11-25 | Ericsson Telefon Ab L M | Method of reducing the amount of transmitted data when a signaling message is sent more than once between two nodes in a TCP / IP based network |
| US6671722B1 (en) * | 1999-07-08 | 2003-12-30 | Intel Corporation | Stack-less, CPU-less creation of valid SNMP-trap packets |
| EP1081910B1 (en) * | 1999-08-06 | 2005-08-31 | Matsushita Electric Industrial Co., Ltd. | Data transmission and reception apparatus |
| US6700888B1 (en) | 1999-09-28 | 2004-03-02 | Telefonaktiebolaget Lm Ericsson (Publ) | Manipulating header fields for improved performance in packet communications |
| US6791982B2 (en) | 1999-09-29 | 2004-09-14 | Telefonaktiebolaget Lm Ericsson | Segmentation protocol that supports compressed segmentation headers |
| US6882637B1 (en) * | 1999-10-14 | 2005-04-19 | Nokia Networks Oy | Method and system for transmitting and receiving packets |
| US7136930B1 (en) | 1999-11-05 | 2006-11-14 | Nokia Corporation | System and method for effective use of air link between mobile stations and gateway servers |
| US6782047B1 (en) * | 1999-11-09 | 2004-08-24 | Nokia Networks Oy | Variable length encoding of compressed data |
| EP1670143B1 (en) * | 1999-11-09 | 2008-12-24 | Nokia Corporation | Variable length encoding of compressed data |
| US6535925B1 (en) | 1999-11-09 | 2003-03-18 | Telefonaktiebolaget L M Ericsson (Publ) | Packet header compression using division remainders |
| DE60018452T2 (en) * | 1999-11-29 | 2005-12-29 | Nokia Corp. | SYSTEM AND METHOD FOR IDENTIFYING CODING / DECODING INFORMATION IN A MOBILE COMMUNICATION NETWORK |
| US6577596B1 (en) * | 1999-11-30 | 2003-06-10 | Telefonaktiebolaget Ln Ericsson (Publ) | Method and apparatus for packet delay reduction using scheduling and header compression |
| US6735186B1 (en) * | 1999-12-22 | 2004-05-11 | Nokia Corporation | Method and system for optimizing usage of air link |
| US20030191876A1 (en) * | 2000-02-03 | 2003-10-09 | Fallon James J. | Data storewidth accelerator |
| US20010047473A1 (en) | 2000-02-03 | 2001-11-29 | Realtime Data, Llc | Systems and methods for computer initialization |
| US6429811B1 (en) | 2000-02-15 | 2002-08-06 | Motorola, Inc. | Method and apparatus for compressing GPS satellite broadcast message information |
| US6999429B1 (en) | 2000-03-03 | 2006-02-14 | Telefonaktiebolaget Lm Ericsson | Access technology integrated header compression |
| US6388584B1 (en) | 2000-03-16 | 2002-05-14 | Lucent Technologies Inc. | Method and apparatus for data compression of network packets |
| GB0007781D0 (en) * | 2000-03-30 | 2000-05-17 | Sony Uk Ltd | Data compression |
| DE10037004B4 (en) * | 2000-07-29 | 2004-01-15 | Sms Demag Ag | Roll stand for belt edge-oriented shifting of the intermediate rolls in a 6-roll stand |
| GB2367459A (en) * | 2000-09-28 | 2002-04-03 | Roke Manor Research | Method of compressing data packets |
| US7417568B2 (en) | 2000-10-03 | 2008-08-26 | Realtime Data Llc | System and method for data feed acceleration and encryption |
| US9143546B2 (en) | 2000-10-03 | 2015-09-22 | Realtime Data Llc | System and method for data feed acceleration and encryption |
| US8692695B2 (en) | 2000-10-03 | 2014-04-08 | Realtime Data, Llc | Methods for encoding and decoding data |
| US6618397B1 (en) * | 2000-10-05 | 2003-09-09 | Provisionpoint Communications, Llc. | Group packet encapsulation and compression system and method |
| DE60120466T2 (en) * | 2000-10-11 | 2007-01-18 | Broadcom Corp., Irvine | Efficient transmission of RTP packets in a network |
| KR100384887B1 (en) * | 2000-10-11 | 2003-05-22 | 주식회사 케이티 | Method for accepting mobile IP in MPLS domain network |
| US6649567B2 (en) * | 2001-10-11 | 2003-11-18 | Isp Investments Inc. | Controlled release microbiocide for porous surfaces |
| US7970886B1 (en) * | 2000-11-02 | 2011-06-28 | Arbor Networks, Inc. | Detecting and preventing undesirable network traffic from being sourced out of a network domain |
| US6883035B2 (en) * | 2000-11-16 | 2005-04-19 | Telefonaktiebolaget Lm Ericsson (Publ) | System and method for communicating with temporary compression tables |
| US6950445B2 (en) * | 2000-11-16 | 2005-09-27 | Telefonaktiebolaget Lm Ericsson (Publ) | Communication system and method for shared context compression |
| US6963587B2 (en) * | 2000-11-16 | 2005-11-08 | Telefonaktiebolaget Lm Ericsson (Publ) | Communication system and method utilizing request-reply communication patterns for data compression |
| US20020064190A1 (en) * | 2000-11-30 | 2002-05-30 | Sikora John J. | Method for compressing packet headers within a trunking protocol for aggregating multiple information channels across a network |
| EP1220508A1 (en) * | 2000-12-29 | 2002-07-03 | TELEFONAKTIEBOLAGET LM ERICSSON (publ) | Method for transmitting data packets in a cellular communication network |
| DE10143102B4 (en) * | 2001-01-08 | 2009-11-19 | Siemens Ag | Method for header compression in video coding |
| WO2002054779A2 (en) * | 2001-01-08 | 2002-07-11 | Siemens Aktiengesellschaft | Method for header compression in a video coding |
| US20020095513A1 (en) * | 2001-01-16 | 2002-07-18 | Freddie Lin | Multilayer lossless data compression across a network |
| US7082569B2 (en) * | 2001-01-17 | 2006-07-25 | Outlooksoft Corporation | Systems and methods providing dynamic spreadsheet functionality |
| GB2372180A (en) * | 2001-02-07 | 2002-08-14 | Motorola Inc | Compression of SIP/SDP headers by omitting standard fields from transmission and insertion of omitted fields from cache at receiver |
| US7386046B2 (en) | 2001-02-13 | 2008-06-10 | Realtime Data Llc | Bandwidth sensitive data compression and decompression |
| US7155173B2 (en) * | 2001-03-14 | 2006-12-26 | Nokia Corporation | Method and system for providing a context for message compression |
| CN1316750C (en) * | 2001-03-29 | 2007-05-16 | 皇家菲利浦电子有限公司 | Reduced data stream for transmitting signal |
| US20020191691A1 (en) * | 2001-05-10 | 2002-12-19 | Holborow Clive Eric | Payload header suppression including removal of fields that vary in known patterns |
| JP3512177B2 (en) * | 2001-05-16 | 2004-03-29 | 松下電器産業株式会社 | Packet receiving apparatus and packet transmitting method |
| US7382878B2 (en) * | 2001-06-22 | 2008-06-03 | Uponus Technologies, Llc | System and method for data encryption |
| US20030009595A1 (en) * | 2001-07-09 | 2003-01-09 | Roger Collins | System and method for compressing data using field-based code word generation |
| WO2003007572A1 (en) * | 2001-07-13 | 2003-01-23 | Roke Manor Research Limited | Method for compressing protocols and related system |
| GB2378361B (en) * | 2001-08-01 | 2004-10-13 | Roke Manor Research | Method for compression of data |
| US7191375B2 (en) * | 2001-12-28 | 2007-03-13 | Intel Corporation | Method and apparatus for signaling an error condition to an agent not expecting a completion |
| US7099318B2 (en) * | 2001-12-28 | 2006-08-29 | Intel Corporation | Communicating message request transaction types between agents in a computer system using multiple message groups |
| US6944617B2 (en) | 2001-12-28 | 2005-09-13 | Intel Corporation | Communicating transaction types between agents in a computer system using packet headers including an extended type/extended length field |
| US7184399B2 (en) * | 2001-12-28 | 2007-02-27 | Intel Corporation | Method for handling completion packets with a non-successful completion status |
| US7581026B2 (en) * | 2001-12-28 | 2009-08-25 | Intel Corporation | Communicating transaction types between agents in a computer system using packet headers including format and type fields |
| US7484007B2 (en) * | 2002-02-01 | 2009-01-27 | Codekko Inc. | System and method for partial data compression and data transfer |
| KR100497357B1 (en) * | 2002-06-26 | 2005-06-23 | 삼성전자주식회사 | Header compressed and packet multiplexed apparatus and method in the network environment based on IP |
| US7324516B2 (en) * | 2002-08-14 | 2008-01-29 | Intel Corporation | Data packet header conversion |
| US20040136476A1 (en) * | 2003-01-10 | 2004-07-15 | Rosen Eric C. | Method and apparatus for compressing header information for short data burst messaging |
| AU2004272192C1 (en) * | 2003-09-10 | 2010-05-06 | Hyperdata Technologies, Inc. | Internet protocol optimizer |
| US20050117604A1 (en) * | 2003-11-19 | 2005-06-02 | Rasmus Villefrance | Transport layer protocol for a peripheral module for a communication device |
| DE102004005467A1 (en) * | 2004-02-04 | 2005-09-01 | Deutsche Telekom Ag | Method and apparatus for accelerating Web / WAP-based access to information |
| US7564381B1 (en) * | 2004-02-16 | 2009-07-21 | Cisco Technology, Inc. | System and method for code-based compression in a communications environment |
| US7924731B2 (en) * | 2004-11-15 | 2011-04-12 | Telefonaktiebolaget Lm Ericsson (Publ) | Method and apparatus for handling out-of-sequence packets in header decompression |
| US7711799B2 (en) * | 2004-11-22 | 2010-05-04 | Alcatel-Lucent Usa Inc. | Method and apparatus for pre-packetized caching for network servers |
| US20060153196A1 (en) * | 2005-01-11 | 2006-07-13 | Conexant Systems, Inc. | Systems and methods for achieving improved ADSL data rates over USB 1.1 channel |
| US8511558B2 (en) * | 2005-04-12 | 2013-08-20 | Sandisk Il Ltd. | Smartcard power management |
| US8804765B2 (en) * | 2005-06-21 | 2014-08-12 | Optis Wireless Technology, Llc | Dynamic robust header compression |
| DE102005062077A1 (en) * | 2005-12-22 | 2007-06-28 | Micronas Gmbh | Method for compressing data, method for decompressing compressed data and devices therefor |
| US7882084B1 (en) | 2005-12-30 | 2011-02-01 | F5 Networks, Inc. | Compression of data transmitted over a network |
| US7577710B2 (en) * | 2006-02-07 | 2009-08-18 | Stauffer John E | System and method for prioritizing electronic mail and controlling spam |
| US8832043B2 (en) | 2006-05-31 | 2014-09-09 | International Business Machines Corporation | Method and system for transformation of logical data objects for storage |
| US8769311B2 (en) | 2006-05-31 | 2014-07-01 | International Business Machines Corporation | Systems and methods for transformation of logical data objects for storage |
| EP2651092B1 (en) * | 2007-03-12 | 2015-08-12 | Citrix Systems, Inc. | System and method of determining a precedence for matching fingerprints in a compression history |
| US7827237B2 (en) * | 2007-03-12 | 2010-11-02 | Citrix Systems, Inc. | Systems and methods for identifying long matches of data in a compression history |
| US7619545B2 (en) * | 2007-03-12 | 2009-11-17 | Citrix Systems, Inc. | Systems and methods of using application and protocol specific parsing for compression |
| US8255570B2 (en) * | 2007-03-12 | 2012-08-28 | Citrix Systems, Inc. | Systems and methods of compression history expiration and synchronization |
| US7865585B2 (en) * | 2007-03-12 | 2011-01-04 | Citrix Systems, Inc. | Systems and methods for providing dynamic ad hoc proxy-cache hierarchies |
| US7532134B2 (en) | 2007-03-12 | 2009-05-12 | Citrix Systems, Inc. | Systems and methods for sharing compression histories between multiple devices |
| US7460038B2 (en) * | 2007-03-12 | 2008-12-02 | Citrix Systems, Inc. | Systems and methods of clustered sharing of compression histories |
| US8391148B1 (en) * | 2007-07-30 | 2013-03-05 | Rockstar Consortion USLP | Method and apparatus for Ethernet data compression |
| US7885294B2 (en) * | 2007-08-23 | 2011-02-08 | Cisco Technology, Inc. | Signaling compression information using routing protocols |
| JP5345154B2 (en) * | 2008-01-11 | 2013-11-20 | テレフオンアクチーボラゲット エル エム エリクソン（パブル） | Message handling in IP multimedia subsystem |
| US8548002B2 (en) * | 2008-02-08 | 2013-10-01 | Koolspan, Inc. | Systems and methods for adaptive multi-rate protocol enhancement |
| JP5815515B2 (en) | 2009-06-25 | 2015-11-17 | コーニンクレッカ フィリップス エヌ ヴェＫｏｎｉｎｋｌｉｊｋｅ Ｐｈｉｌｉｐｓ Ｎ．Ｖ． | Method and apparatus for processing data packets |
| US8838544B2 (en) * | 2009-09-23 | 2014-09-16 | International Business Machines Corporation | Fast history based compression in a pipelined architecture |
| US20120113824A1 (en) * | 2010-11-05 | 2012-05-10 | Telefonaktiebolaget L M Ericsson (Pub) | Report sequence with performance data transmitted between two nodes of a comunication system |
| KR20120054699A (en) * | 2010-11-22 | 2012-05-31 | 삼성전자주식회사 | Memory controller, data storage system including the same and method thereof |
| KR101891981B1 (en) * | 2011-10-07 | 2018-10-04 | 한국전자통신연구원 | System and method for analysing of online game packets |
| JP5906078B2 (en) * | 2011-12-20 | 2016-04-20 | キヤノン株式会社 | Data transfer apparatus and data transfer method |
| US8786471B1 (en) | 2013-03-15 | 2014-07-22 | Oracle International Corporation | Lossless data compression with variable width codes |
| FR3006534B1 (en) * | 2013-05-31 | 2015-05-22 | Thales Sa | METHODS OF TRANSMITTING AND RECEIVING DATA BETWEEN A TERMINAL AND A GATEWAY, IN PARTICULAR THROUGH A SATELLITE LINK |
| CN104009921B (en) * | 2014-04-28 | 2017-09-19 | 开网科技（北京）有限公司 | The data message forwarding method matched based on arbitrary fields |
| US10140033B2 (en) * | 2015-06-15 | 2018-11-27 | Xitore, Inc. | Apparatus, system, and method for searching compressed data |
| CN113364508B (en) * | 2021-04-30 | 2022-08-16 | 深圳震有科技股份有限公司 | Voice data transmission control method, system and equipment |
| FR3122960A1 (en) * | 2021-05-17 | 2022-11-18 | Sigfox | Optimization of the geolocation of a terminal from one or more identifiers of neighboring transmitting devices |
| CN115334169B (en) * | 2022-04-28 | 2023-06-06 | 深圳证券通信有限公司 | Communication protocol coding method capable of saving network bandwidth |
| CN115065725B (en) * | 2022-05-12 | 2024-01-12 | 北京奇艺世纪科技有限公司 | Data processing method, device, electronic equipment and storage medium |
| CN116193113A (en) * | 2022-12-07 | 2023-05-30 | 辉羲智能科技(上海)有限公司 | Data decompression and compression method and device |
| CN116436987B (en) * | 2023-06-12 | 2023-08-22 | 深圳舜昌自动化控制技术有限公司 | IO-Link master station data message transmission processing method and system |
Family Cites Families (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US4464650A (en) * | 1981-08-10 | 1984-08-07 | Sperry Corporation | Apparatus and method for compressing data signals and restoring the compressed data signals |
| US4814746A (en) * | 1983-06-01 | 1989-03-21 | International Business Machines Corporation | Data compression method |
| US4558302A (en) * | 1983-06-20 | 1985-12-10 | Sperry Corporation | High speed data compression and decompression apparatus and method |
| US4612532A (en) * | 1984-06-19 | 1986-09-16 | Telebyte Corportion | Data compression apparatus and method |
| US4730348A (en) * | 1986-09-19 | 1988-03-08 | Adaptive Computer Technologies | Adaptive data compression system |
| JPH0815263B2 (en) * | 1986-12-12 | 1996-02-14 | 株式会社日立製作所 | Data compression / decompression method |
| US4899148A (en) * | 1987-02-25 | 1990-02-06 | Oki Electric Industry Co., Ltd. | Data compression method |
| US4881075A (en) * | 1987-10-15 | 1989-11-14 | Digital Equipment Corporation | Method and apparatus for adaptive data compression |
| US4847619A (en) * | 1987-10-19 | 1989-07-11 | Hewlett-Packard Company | Performance-based reset of data compression dictionary |
| US4899147A (en) * | 1988-06-03 | 1990-02-06 | Unisys Corporation | Data compression/decompression apparatus with throttle, start-up and backward read controls |
| US5126739A (en) * | 1989-01-13 | 1992-06-30 | Stac Electronics | Data compression apparatus and method |
| US5045852A (en) * | 1990-03-30 | 1991-09-03 | International Business Machines Corporation | Dynamic model selection during data compression |
| US5023610A (en) * | 1990-06-13 | 1991-06-11 | Cordell Manufacturing, Inc. | Data compression method using textual substitution |
| US5049881A (en) * | 1990-06-18 | 1991-09-17 | Intersecting Concepts, Inc. | Apparatus and method for very high data rate-compression incorporating lossless data compression and expansion utilizing a hashing technique |
-
1992
- 1992-04-08 CA CA002065578A patent/CA2065578C/en not_active Expired - Lifetime
-
1993
- 1993-05-27 US US08/068,284 patent/US5293379A/en not_active Expired - Fee Related
Also Published As
| Publication number | Publication date |
|---|---|
| US5293379A (en) | 1994-03-08 |
| CA2065578A1 (en) | 1992-10-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CA2065578C (en) | Packet-based data compression method | |
| CA2334497C (en) | Method and apparatus for data compression of network packets employing per-packet hash tables | |
| US7403136B2 (en) | Block data compression system, comprising a compression device and a decompression device and method for rapid block data compression with multi-byte search | |
| US5389922A (en) | Compression using small dictionaries with applications to network packets | |
| US5253325A (en) | Data compression with dynamically compiled dictionary | |
| US7643505B1 (en) | Method and system for real time compression and decompression | |
| JP3900435B2 (en) | Data packet routing method and routing apparatus | |
| US6121903A (en) | On-the-fly data re-compression | |
| US5841376A (en) | Data compression and decompression scheme using a search tree in which each entry is stored with an infinite-length character string | |
| EP1134900B1 (en) | Method and apparatus for data compression of network packets | |
| US5831558A (en) | Method of compressing and decompressing data in a computer system by encoding data using a data dictionary | |
| EP0903866B1 (en) | Method and apparatus for data compression | |
| EP0793349A2 (en) | Method and apparatus for performing data compression | |
| WO1998004045A1 (en) | Data compression and decompression system with immediate dictionary updating interleaved with string search | |
| EP0903865A1 (en) | Method and apparatus for compressing data | |
| US8909813B2 (en) | Efficient processing of compressed communication traffic | |
| US6289130B1 (en) | Method for real-time lossless data compression of computer data | |
| US6240213B1 (en) | Data compression system having a string matching module | |
| KR101791877B1 (en) | Method and apparatus for compressing utf-8 code character | |
| WO2000038098A1 (en) | Synthetic history for adaptive data compression | |
| GB2360916A (en) | Compression encoder which transmits difference between new data word and recent data word where this falls within a threshold | |
| Acharya et al. | An On-line Variable Length Binary Encoding | |
| WO2003007572A1 (en) | Method for compressing protocols and related system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| EEER | Examination request | ||
| MKEX | Expiry |