|Publication number||USRE39925 E1|
|Application number||US 10/824,613|
|Publication date||Nov 27, 2007|
|Filing date||Apr 15, 2004|
|Priority date||Dec 15, 2000|
|Also published as||CN1148005C, CN1359196A, US6373412|
|Publication number||10824613, 824613, US RE39925 E1, US RE39925E1, US-E1-RE39925, USRE39925 E1, USRE39925E1|
|Inventors||Joan L. Mitchell, Albert N. Cazes, Neil M. Leeder|
|Original Assignee||International Business Machines Corporation|
|Export Citation||BiBTeX, EndNote, RefMan|
|Patent Citations (8), Referenced by (6), Classifications (18), Legal Events (3)|
|External Links: USPTO, USPTO Assignment, Espacenet|
This application is related to U.S. patent applicaton Ser. No. 09/736,444 filed concurrently herewith, assigned to the Assignee of the present invention and which is hereby fully incorporated by reference.
1. Field of the Invention
The present invention generally relates to image compression for diverse applications and, more particularly, in combination with a structure for storing Discrete Cosine Transform (DCT) blocks in a packed format, performing Huffman entropy encoding and decoding in accordance with the JPEG (Joint Photographic Experts Group) standard.
2. Description of the Prior Art
Pictorial and graphics images contain extremely large amounts of data and, if digitized to allow transmission or processing by digital data processors, often requires many millions of bytes to represent respective pixels of the image or graphics with good fidelity. The purpose of image compression is to represent images with less data in order to save storage costs or transmission time and costs. The most effective compression is achieved by approximating the original image, rather than reproducing it exactly. The JPEG standard, discussed in detail in “JPEG Still Image Data Compression Standard” by Pennebaker and Mitchell, published by Van Nostrand Reinhold, 1993, which is hereby fully incorporated by reference, allows the interchange of images between diverse applications and opens up the capability to provide digital continuous-tone color images in multi-media applications.
JPEG is primarily concerned with images that have two spatial dimensions, contain gray scale or color information, and possess not temporal dependence, as distinguished from the MPEG (Moving Picture Experts Group) standard. JPEG compression can reduce the storage requirements by more than an order of magnitude and improve system response time in the process. A primary goal of the JPEG standard is to provide the maximum image fidelity for a given volume of data and/or available transmission or processing time and any arbitrary degree of data compression is accommodated. It is often the case that data compression by a factor of twenty or more (and reduction of transmission or processing time by a comparable factor) will not produce artifacts which are noticeable to the average viewer.
One of the basic building blocks for JPEG is the Discrete Cosine Transform (DCT). An important aspect of this transform is that it produces uncorrelated coefficients. Decorrelation of the coefficients is very important for compression because each coefficient can be treated independently without loss of compression efficiency. Another important aspect of the DCT is the ability to quantize the DCT coefficients using visually-weighted quantization values. Since the human visual system response is very dependent on spatial frequency, by decomposing an image into a set of waveforms, each with a particular spatial frequency, it is possible to separate the image structure the eye can see from the image structure that is imperceptible. The DCT thus provides a good approximation to this decomposition to allow truncation or omission of data which does not contribute significantly to the viewer's perception of the fidelity of the image.
In accordance with the JPEG standard, the original monochrome image is first decomposed into blocks of sixty-four pixels in an 8×8 array at an arbitrary resolution which is presumably sufficiently high that visible aliasing is not produced. (Color images are compressed by first decomposing each component into an 8×8 pixel blocks separately.) Techniques and hardware is known which can perform a DCT on this quantized image data very rapidly, yielding sixty-four DCT coefficients. Many of these DCT coefficients for many images will be zero (which do not contribute to the image in any case) or near-zero which can be neglected or omitted when corresponding to spatial frequencies to which the eye is relatively insensitive. Since the human eye is less sensitive to very high and very low spatial frequencies, as part of the JPEG standard, providing DCT coefficients in a so-called zig-zag pattern which approximately corresponds to an increasing sum of spatial frequencies in the horizontal and vertical directions tends to group the DCT coefficients corresponding less important spatial frequencies at the ends of the DCT coefficient data stream, allowing them to be compressed efficiently as a group in many instances.
While the above-described discrete cosine transformation and coding may provide significant data compression for a majority of images encountered in practice, actual reduction in data volume is not guaranteed and the degree of compression is not optimal, particularly since equal precision for representation of each DCT coefficient would require the same number of bits to be transmitted (although the JPEG standard allows for the DCT values to be quantized by ranges that are coded in a table). That is, the gain in compression developed by DCT coding derives largely from increased efficiency in handling zero and near-zero values of the DCT coefficients although some compression is also achieved through quantization that reduces precision. Accordingly, the JPEG standard provides a second stage of compression and coding which is known as entropy coding.
The concept of entropy coding generally parallels the concept of entropy in the more familiar context of thermodynamics where entropy quantifies the amount of “disorder” in a physical system. In the field of information theory, entropy is a measure of the predictability of the content of any given quantum of information (e.g. symbol) in the environment of a collection of data of arbitrary size and independent of the meaning of any given quantum of information or symbol. This concept provides an achievable lower bound for the amount of compression that can be achieved for a given alphabet of symbols and, more fundamentally, leads to an approach to compression on the premise that relatively more predictable data or symbols contain less information than less predictable data or symbols and the converse that relatively less predictable data or symbols contain more information than more predictable data or symbols. Thus, assuming a suitable code for the purpose, optimally efficient compression can be achieved by allocating fewer bits to more predictable symbols or values (that are more common in the body of data and include less information) while reserving longer codes for relatively rare symbols or values.
As a practical matter, Huffman coding and arithmetic coding are suitable for entropy encoding and both are accommodated by the JPEG standard. One operational difference for purposes of the JPEG standard is that, while tables of values corresponding to the codes are required for both coding techniques, default tables are provided for arithmetic coding but not for Huffman coding. However, some particular Huffman tables, although they can be freely specified under the JPEG standard to obtain maximal coding efficiency and image fidelity upon reconstruction, are often used indiscriminately, much in the nature of a default, if the image fidelity is not excessively compromised in order to avoid the computational overhead of computing custom Huffman tables.
It should be appreciated that while entropy coding, particularly using Huffman coding, guarantees a very substantial degree of data compression if the coding or conditioning tables are reasonably well-suited to the image, the encoding, itself, is very computationally intensive since it is statistically based and requires collection of statistical information regarding a large number of image values or values representing them, such as DCT coefficients. Conversely, the use of tables embodying probabilities which do not represent the image being encoded could lead to expansion rather than compression if the image being encoded requires coding of many values which are relatively rare in the image from which the tables were developed even though such a circumstance is seldom encountered.
It is for this reason that some Huffman tables have effectively come into standard usage even though optimal compression and/or optimal fidelity for the degree of compression utilized will not be achieved. Conversely, compression efficiency of Huffman ending can usually be significantly increased and greater image fidelity optimally maintained for a given number of bits of data by custom Huffman tables corresponding to the image of interest but may be achievable only with substantial computational burden for encoding.
Another inefficiency of Huffman coding characteristically is encountered when the rate of occurrence of any particular value to be encoded rises above 50% due to the hierarchical nature of the technique of assigning Huffman codes to respective coded values and the fact that at least one bit must be used to represent the most frequently occurring and most predictable value even though the information contained therein may, ideally, justify less. For example, if the rate of occurrence of a single value rises to 75%, the efficiency of Huffman coding drops to 86%. As a practical matter, the vast majority of DCT coefficients are (or are quantized to) zero and substantial inefficiencies are therefore frequently encountered in Huffman encoding of DCT coefficients or DCT coefficient differences. For the AC coefficients, the JPEG committee solved this problem by collecting runs of zero-valued coefficient together and not coding the individual zero-valued coefficients. Thus the likelihood of a Huffman symbol occurring more than half the time is greatly reduced.
Huffman decoding also requires a substantial computational burden since the compression efficiency derives largely from a variable length code that requires additional processing to detect the ending point of respective values or symbols. Additionally, this processing is achieved through use of coding tables which must be transmitted with the entropy coded image data and may change with every image. Complexity of access to data in tables is aggravated by the fact that Huffman codes are of variable length in order to allocate numbers of bits in accordance with the predictability of the data or amount of data contained in a particular symbol or value. To increase response speed it has been the practice to compute and store look-up tables from the Huffman tables which can then be used for decoding. Therefore, at least two bytes of table data must often be accessed per code word.
Moreover, since sixteen bit Huffman code lengths are allowed, a prior art method of decoding would access a Huffman table with a 216 entries for the symbol values and 216 entries for the code length. These entries must be computed each time a Huffman table is specified or changed. Up to fours DC and four AC tables could be needed to decode and interleaved baseline four-component image. For hardware with small on-chip caching or RAM, such large tables will degrade performance because extra processing cycles are needed with every cache miss or RAM access.
It is therefore an object of the present invention to provide a technique of Huffman encoding and decoding which can be accomplished in much reduced time with reduced processing resources and hardware.
It is another object of the invention to provide a technique of Huffman encoding and decoding which is enhanced through use of a packed intermediate data format.
In order to accomplish these and other objects of the invention, a method of Huffman encoding symbols is provided comprising steps of defining a seed value for the first occurrence of a code of a given length in a table, storing a length of a code word, storing the length and the code word in a first format when a number of bits of said number and said code word are less than or equal to a predetermined number of bits, and storing an index to the seed value, an offset and the code word in a second format when the number and the image data comprise a number of bits greater than the predetermined number of bits.
In accordance with another aspect of the invention, a method of Huffman decoding compressed data is provided including steps of testing bits of a data stream with each of a plurality of test criteria to determine a length of a valid Huffman code, combining one of a plurality of offsets corresponding to the length with the valid Huffman code to form an index, and accessing a symbol value in a Huffman table using the index.
The foregoing and other objects, aspects and advantages will be better understood from the following detailed description of a preferred embodiment of the invention with reference to the drawings, in which:
Referring now to the drawings, and more particularly in
The marker segment in accordance with the JPEG standard, as shown in
The next sixteen bytes specify the number of codes of each hit length, Li, that will be included in the following table to correspond to the maximum bit length allowed under the JPEG standard (but is otherwise arbitrary). As a notation convention, the number of one-bit codes is 1.1 and the number of sixteen-bit codes is L16. Not all bit lengths will be represented, particularly if the image precision is eight bits which, after quantization yields coefficients of, at most, eleven bits and all code lengths which are not represented in image data will be coded as hexidecimal “00”. The following bytes, V1.1 to V16, t.16 are the actual values corresponding to each Huffman code, in order of the respective frequency/likelihood of occurrence (including instances where more than one code of a given length are provided).
It should be recalled that entropy coding such as Huffman coding assigns the fewer bits to values with greater frequency or likelihood of occurrence and many of the DCT coefficients could be zero. Therefore, the JPEG standard provides for many as eight Huffman tables each having as many as 256 entries (and assures that the mapping of particular Huffman codes to particular DCT coefficients are always defined). To speed this process, look-up tables may also be provided to index into the Vij fields of the Huffman tables which may reduce the number of operations to find a particular Vij value but requires additional memory resources and presents an additional computational burden to develop larger numbers of table entries, as alluded to above.
Application of the marker segment syntax will now be explained in connection with the exemplary Huffman tables of FIG. 2. As introduction it should be understood that the result of quantization and discrete cosine transformation (DCT) in the compression process is a sequence of DCT coefficients in raster scan order. Zero-valued coefficients are roughly grouped by accessing them in zig-zag order and further compression (and enhancement of other transformation processes including entropy coding) obtained by packing data in accordance with run and size (R/S) data as disclosed in the above incorporated, concurrently filed related application. Runs are the length of consecutive strings of zero-valued AC DCT coefficients and the size is the number of bits necessary to specify a non-zero DCT coefficient. The run or R value is coded as the more significant four-bit nibble of a byte and the size or S value is specified as the less significant four-bit nibble of the same R/S byte.
It should be recognized that DC DCT coefficient differences and AC DCT coefficient values are not fully specified by the R/S values. However, encoding only the R/S vales in accordance with the JPEG standard and concatenating or appending to each code the actual bits of a positive AC DCT coefficient or DC DCT coefficient difference values and bits modified according to a suitable convention for negative AC DCT coefficient or DC DCT coefficient difference values as shown in
The first marker in
The third marker is “FFC4” which represents the beginning of a marker segment specifying one or more Huffman tables. The following two bytes “01 A2” are hexadecimal code for four hundred eighteen which is the number of bytes prior to the next marker “FFDA” representing start of scan (SOS) where the length of a parameter list is specified beginning with the number of components (e.g. R, G or B, grey scale, luminance/chrominance, etc., and followed by compressed data schematically indicated by “ . . . ”.
The next byte contains two nibbles representing Tc and Th and defining the table as DC table 0 since both nibbles are zero. Since these are DC tables and only one DC DCT coefficient is provided per block, there are no run lengths and the first nibble of each byte is zero while the second nibble specifies the number of extra bits of each size (which is the number of extra bits following next in the compressed data stream). The particular numbers of codes corresponds to the Huffman codes shown in
At this point, only 29 bytes (1 for Tc, Th, 16 for Li and 12 for Vij) have been used which is much less than four hundred eighteen bytes of parameter length specified immediately after the marker. Therefore the two nibbles of the next byte define a class and name for the second Huffman table specified in this marker segment, in this case, AC table 0 since the Tc and Th values are 1 and 0, respectively. The following sixteen bytes are Li values indicating the number of codes of each length and the next following bytes, equal in number to the sum of the codes enumerated in the Li bytes to a maximum of sixty-four, are Vij symbol values for respective ones of the Huffman codes in descending order of frequency of occurrence to complete AC table 0.
The remainder of the four hundred eighteen are comprised of two more tables, DC table 1 and AC table 1, delineated and coded in the same format discussed above. The Huffman codes for DC table 1 are tabulated in FIG. 4.
From a comparison of
It will be recalled from the foregoing that The structure of the Huffman tables allows the correspondence of particular Huffman codes and the R/S values respectively assigned thereto to be reconstructed. However, as alluded to above, such a reconstruction is complex and time-consuming and it is much more common to use look-up tables to index into the Huffman tables. This process is complicated by the fact that Huffman codes are of variable length and the number (sixteen or fewer) of consecutive bits which corresponds to a valid Huffman code must also be recognized.
A known method of recognizing a valid Huffman code and indexing into the Huffman tables would be to take sixteen (next) consecutive bits in the compressed data and look up (in a table having 216 entries) the number of bits in the code and, in another table (of similar size), look up the R/S byte corresponding to the code. However, this is very wasteful in terms of storage space as well as causing a substantial memory burden. For example, if the code was a one-bit code (at the beginning of the sixteen bits examined, half of each table would have a “1” for the number of bits and the same R/S value byte for the symbol. Other known methods use multiple tables and require a correspondingly greater number of look-up operations.
The invention exploits the characteristic of Huffman codes that each code can be considered to have a numeric value for any given number of binary bits and that a numeric value that is too large for a given number of bits can be calculated. If n bits of compressed data are logically greater than or equal to this maximum value (hereinafter max(n) where n=number of bits), an additional bit must be examined and compared to max(n+1). The invention also explains the fact that, in terms of hardware architecture, these tests can be carried out in parallel since all tests will pass (e.g., the logical value is too large for a valid Huffman code) until the correct number of bits is considered and the remainder of the tests on larger numbers of bits will fail.
For example, as shown in
That is, the first test compares the current most significant bit in the C-register (MSB(C)) against max(1), a one bit value. If there are no one-bit codes, max(1)=0. If there is a one-code (which must be “0” since all “1” codes are disallowed) max(1)=1. There will be a non-zero max(n) for the first Ln that has some codes since max(n) values are constructed such that max(n) is the first code of n bits that is the prefix to a longer code word (e.g. max(2)=“10” which is the prefix of category 5 of FIG. 3).
More generically, max(i) and offset (i) in terms of Li are computed as follows:
Offset (i) is used to determine where in the list of R/S values the decoded value corresponding to the code is located. That is, if i bits of the Huffman code are added to offset(i), the resulting value will be the address in the Huffman table where the corresponding R/S byte will be found. This provides several very important advantages in the context of the invention. Specifically, the function of a plurality of look-up tables, each having 216 entries are replaced by tables of max(n) and offset(n) values limited to sixteen entries and which can be very rapidly computed and efficiently stored. Since this table is singular and small, data therein may be retrieved very rapidly to provide a direct index into the Huffman tables to obtain R/S data that is used to evaluate the DCT coefficient and coefficient difference data included with the Huffman codes in the data stream in the JPEG standard syntax (which can be utilized with other processing techniques and for other types of data).
In summary, the decoding of compressed data is illustrated in
to present this data in the format of
A perfecting feature of the process of
The max(n) values are tightly packed into words as follows:
XC_max<0>=max(1),max(2), max(3),max(4),max(5), max (6),max(7),0000=32bits;
xC_max<3>=max(13),max(14),00000=32 bits; and
These values are manipulated as depicted in
The above description of the decoder and decoding process in accordance with the invention is thus seen to provide very significant increase of speed and responsiveness in reconstructing image data compressed in accordance with the JPEG standard and, by the same token, substantial reductions in data processing hardware and/or software requirements. Significant advantages are also provided in the encoding process in combination with the packet data format disclosed in the above-incorporated, concurrently filed application. The intermediate data format disclosed therein is illustrated in FIG. 10.
During encoding R/S bytes must be converted to a Huffman code. The packed data format of
Then, as long as i is less than 16, i is incremented by 1, the seed is doubled and stored in H_seed(i). The code is set to seed and then seed is incremented by a number of i-bit codes and j is initialized to zero.
As long as j is less than the number of codes Li, R/S is set to the next Vij symbol value in the Define Huffman Table (DHT) marker segment. If the number of bits in the code is less than six, the number of bits and the code are packed into H_code(RS) with three most significant bits in the byte set to the number of bits in the code and the low order five bits set to the code. If i is greater than five, the H_code(RS) is set to the number of bits. Then H. . .offset(RS) is set to j, since this is the j+1th code of the i-bit codes. Both j and code are incremented for both paths.
For software implementations, the H_seed table entries are likely to be implemented as integers of at least sixteen bits since the longest seed is sixteen bits. For hardware, the nth entry only needs to be n bits because it is the seed for codes of n bits. For simplicity in
Each DC and AC Huffman table needs a set of H_seed, H_code and H_offset tables. Fortunately, for baseline implementations, the DC H_code and H_offset tables need only twelve entries since R=0 and the maximum value of S is 11. For generic implementations, the maximum value of S is 16, making a maximum of seventeen entries for these tables.
For simplicity, the H_code and H_offset tables are likely to have 256 entries each because most of the R/S bytes may be used. The entries in these tables are, at most, a byte. Therefore, instead of needing a 16-bit code word entries plus more bits to signify the number of bits in the code words, a small H_seed table is used for the longer code words and only an extra byte is needed to calculate the final code word of that size. All of the short codes (i.e. less than five bits) are contained in the H_code byte. Since the more frequent symbols will have shorter codes, a single byte access to storage contains everything needed for the most commonly coded symbols.
In the prior art, n and the bit pattern of up to 16 bits are stored for up to 256 R/S symbols. In accordance with the invention, however n and the code word are stored in a byte when both can be stored in a total of eight bits or less. This allows the Huffman tables to be considerably shortened and is particularly advantageous since most of the codes encountered will be short enough for this format to be used. Use of this format is indicated by a non-zero value in the three MSB locations of the byte.
For all longer bit patterns, only a length indication is stored as S and the high order nibble is set to 0000 to indicate a different format. For these longer codes, the length is used as an index to the seed (i.e. the first code value of a particular bit length). A corresponding offset (which fits in a byte since the maximum offset is 255) can then be determined from a look-up table to add to the seed to create the code value. After the R/S byte is encoded and stored, the extra S bits are encoded, preferably by shifting to develop a variable length from the two bytes of each AC coefficient value in the intermediate data format of FIG. 10.
In view of the foregoing, it is seen that the invention provides increased efficiency of data processing for encoding, and greatly increased efficiency of computation and storage of tables for indexing into the Huffman tables. These advantages are produced in a manner which is fully compliant with the JPEG standard and are applicable to all applications of Huffman coding.
While the invention has been described in terms of a single preferred embodiment in terms of the JPEG standard, those skilled in the art will recognize that the invention can be practiced with modification within the spirit and scope of the appended claims. For example, these fast encoding and decoding methods apply to any Huffman table which is constructed from the number of codes of each length in ascending order. Further, those skilled in the art will be enabled from the above description of the invention to apply the above-described techniques to Huffman tables constructed in descending order.
|Cited Patent||Filing date||Publication date||Applicant||Title|
|US4396906 *||Nov 17, 1980||Aug 2, 1983||Sri International||Method and apparatus for digital Huffman encoding|
|US5227789 *||Sep 30, 1991||Jul 13, 1993||Eastman Kodak Company||Modified huffman encode/decode system with simplified decoding for imaging systems|
|US5550541 *||Apr 1, 1994||Aug 27, 1996||Dolby Laboratories Licensing Corporation||Compact source coding tables for encoder/decoder system|
|US5652582 *||Dec 29, 1995||Jul 29, 1997||Xerox Corporation||Method of high speed Huffman coding and decoding of lab color images|
|US5686915 *||Dec 27, 1995||Nov 11, 1997||Xerox Corporation||Interleaved Huffman encoding and decoding method|
|US5808570 *||Nov 14, 1996||Sep 15, 1998||Philips Electronics North America Corp.||Device and method for pair-match Huffman transcoding and high-performance variable length decoder with two-word bit stream segmentation which utilizes the same|
|US6040790 *||May 29, 1998||Mar 21, 2000||Xerox Corporation||Method of building an adaptive huffman codeword tree|
|US6130630 *||Oct 27, 1998||Oct 10, 2000||Hewlett-Packard Company||Apparatus and method for compressing Huffman encoded data|
|Citing Patent||Filing date||Publication date||Applicant||Title|
|US7394411 *||Jan 9, 2007||Jul 1, 2008||International Business Machines Corporation||Fast implementation of decoding function for variable length encoding|
|US7864081||Jun 9, 2008||Jan 4, 2011||International Business Machines Corporation||Fast implementation of decoding function for variable length encoding|
|US8073270||Sep 16, 2008||Dec 6, 2011||Silicon Motion, Inc.||Image decoding apparatus and method|
|US20080165038 *||Jan 9, 2007||Jul 10, 2008||International Business Machines Corporation||Fast implementation of decoding function for variable length encoding|
|US20080238734 *||Jun 9, 2008||Oct 2, 2008||International Business Machines Corporation||Fast Implementation Of Decoding Function For Variable Length Encoding|
|US20100067811 *||Mar 18, 2010||Silicon Motion, Inc.||Image decoding apparatus and method|
|U.S. Classification||341/65, 341/51, 341/61|
|International Classification||G06T9/00, H04N7/30, H04N1/419, H03M7/40, H03M7/42|
|Cooperative Classification||H04N19/60, H04N19/91, G06T9/007, H03M7/40, G06T9/005|
|European Classification||G06T9/00S, G06T9/00T, H03M7/40, H04N7/26A4V, H04N7/30|
|Sep 2, 2009||FPAY||Fee payment|
Year of fee payment: 8
|Nov 22, 2013||REMI||Maintenance fee reminder mailed|
|Apr 16, 2014||LAPS||Lapse for failure to pay maintenance fees|