l
METHOD AND APPARATUS FOR
IMPROVED VIDEO CODING USING A
CENTER-BIASED ORTHOGONAL SEARCH
TECHNIQUE AND A ZERO BLOCK
PREDICTOR MODULE
This is a division, of application Ser. No. 08/590,646, filed Jan. 24,1996, abandoned which is an application under 111(a) filed not more than 12 months after prior provisional application Ser. No. 60/002,528, filed Aug. 18, 1995.
TECHNICAL FIELD OF THE INVENTION
This invention relates to image processing and in particular to a method and system for improved encoding of video data.
BACKGROUND OF THE INVENTION
Image compression and decompression are used in a wide range of applications including video conferencing systems, video phones, and motion picture transmission. The conventional approach in these applications has been to use dedicated hardware for video coding, i.e., image compression and decompression. The use of dedicated hardware, typically digital signal processors (DSPs), is required because the video coding process is computationally expensive and slow on general-purpose hardware. As a result, widespread use of these applications has been stymied by the costs associated with deploying the specialized hardware required to achieve good performance. It has been predicted, however, that these applications, in particular video conferencing, will become desktop commodities in the next few years. Improvements to the video coder and the video coding process are necessary to make this prediction a reality.
A diagram illustrating a video coding process in accordance with International Telecommunication Union (ITU) standard H.263, hereinafter referred to as the ITU H.263 standard, for video coding and encoding at very low bitrates, such as at 28K bits per second, is shown in FIG. 1.
The video encoder shown in FIG. 1 includes a color transform module 12, a motion detector module 14, a motion compensation module 16, a transform module 18, a quantization module 20, and a coding module 22. Also included is a feedback module 24 which includes an inverse quantization module 30, an inverse transformation module 28, and a frame reconstruction module 26.
Avideo decoder, as shown in FIG. 2, performs the reverse process of the video coder and includes a bit-stream decoding module 40, the inverse quantization module 30, the inverse transform module 28, an inverse motion compensation module 42, and the frame reconstruction module 26.
In the video coding process, motion compensation performed by the motion compensation module 16 is the most time consuming phase. The transformation and quantization phases, performed by the transformation module 18 and the quantization module 20, respectively, are also expensive phases to perform.
However, with processor speeds doubling every two years, it is possible for software-only solutions to attain good performance and quality and to lower the costs associated with applications which require video processing enough make image processing a commodity item in desktop computing environments.
To overcome the computational requirements of the various stages, the video processing applications in prior art
2
systems employ dedicated DSPs to make the various computationally expensive stages execute faster. Use of dedicated hardware is a weakness of current video conferencing systems. Designing new hardware as the video coding 5 standards change and evolve is expensive, time-consuming and substantially increases the cost of delivered systems. Not only do the high costs associated with dedicated hardware present a barrier against image processing applications becoming a desktop commodity solutions, but they also go against the latest trend in hardware/software solution to use open systems.
Thus, what is needed is a method and system to overcome the limitations and weaknesses of current video processing application implementations. In particular, what is needed is a method and system for video encoding which is compu15 tationally more efficient than those of the prior art and which are amenable to implementation using low-cost generalpurpose DSPs or software-only solutions.
SUMMARY OF THE INVENTION
20 The present invention includes a video coding process which can be implemented on low-cost, general purpose DSPs or as a software only solution on a general purpose microprocessor to get acceptable performance. The present invention modifies the overall video coding process and
25 augments it with optimizations which reduce the overall video coding computation time thereby making it viable to use the low-cost DSP solutions or a software-only solution, both of which permit acceptable performance on today's desktop CPUs.
One aspect of the present invention includes optimizations of the motion detection stage.
Another aspect of the present invention includes optimizations of the motion compensation stage.
Yet another aspect of the present invention includes the
35 addition of a zero block prediction stage.
These and other features of the invention that will be apparent to those skilled in the art from the following detailed description of the invention, taken together with the accompanying drawings.
40 DESCRIPTION OF THE DRAWINGS
FIG. 1 is a block diagram depicting a prior art video encoder;
FIG. 2 shows a block diagram illustrating a prior art video
, decoder,
45
FIG. 3 illustrates a first embodiment of a video encoder in accordance with the present invention;
FIG. 4 depicts a second embodiment of a video encoder in accordance with the present invention; 50 FIG. 5 shows a third embodiment of a video encoder in accordance with the present invention;
FIG. 6 is a flow diagram illustrating the operation of a video encoder in accordance with the present invention;
FIG. 7 shows a flow diagram illustrating the operation of 55 a modified motion detector module in accordance with the present invention;
FIG. 8 illustrates a flow diagram showing the operation of a modified motion compensation module in accordance with the present invention; and 60 FIG. 9 is a block diagram depicting the operation of a zero block predictor module in accordance with the present invention.
DETAILED DESCRIPTION OF THE 65 INVENTION
Avideo coder in accordance with the present invention, as shown in FIG. 3, includes several improvements to the video
3
coder and video coding process of the prior art, shown in FIG. 1. The improvements shown in the video coder in accordance with the present invention as depicted in FIG. 3 provide for increased performance over video coders and video coding processes of the prior art. It is contemplated 5 that the video coder and video coding process in accordance with the present invention are useful in any application using video coding techniques.
The present invention is implemented in C++ using Microsoft's Visual C++ compiler on a workstation which 1° included a Pentium processor. It is contemplated, however, that use of other languages and hardware configurations would be apparent to those skilled in the art.
The overall structure of the video coder in accordance with the present invention is shown in FIG. 3. As discussed 15 hereinabove, the motion compensation stage and the transformation stage are among the most time consuming stages in a typical video coding process. The present invention provides for improvements to the motion detection stage as performed by modified motion detector module 50. The 20 modified motion detector module 50 in accordance with the present invention determines whether to bypass the motion compensation stage performed by the modified motion compensation module 52. The modified motion compensation module 52 also includes improvements to the motion com- 25 pensation stage over that of the motion compensation module 16 of the prior art.
The video coding process in accordance with the present invention also includes the addition of a zero block predic
30
tion stage, performed by zero block predictor module 54. The zero block predictor module 54 determines whether or not to bypass the transformation and quantization stages. Each of these features are discussed in detail hereinbelow.
It is contemplated that other embodiments, in addition to 3J that shown in FIG. 3, of the present invention are possible which also provide improvements to the video coder of the prior art. Another embodiment of the video coder in accordance with the present invention is shown in FIG. 4, which includes the modified motion compensation module 52. 4Q FIG. 5 illustrates yet another embodiment of video coder in accordance with the present invention which includes the modified motion detector module 50 and the zero block predictor module 54.
The operation of the video encoder shown in FIG. 3 is 45 illustrated in FIG. 6. At block 60, the color transformation module 12 generates data signals from the video image signals input at 51. The data signals are then input to the modified motion detector module 50 as shown at block 62 which determines whether movement is detected in the 50 represented image. If, at decision block 64, movement is detected, the modified motion compensation module 52, as shown at block 66, compensates for that movement and operation continues at block 68. If, at decision block 64, movement is not detected, the motion compensation stage 55 performed by the modified motion compensation module 52 is bypassed and operation continues at block 68.
At block 68, the zero block predictor module 54 determines whether the currently processing macroblock of data signals will generate a zero-valued macroblock after the 60 transformation and quantization stages performed by the transform module 18 and the quantization module 20, respectively. If, at decision block 70, a zero-valued macroblock is predicted, the transformation and quantization stages are bypassed and operation continues at block 78. 65
If, at decision block 70, a zero-valued macroblock is not predicted, operation continues at block 72 where the data
4
signals are transformed by the transform module 18. In the present invention, the transform module 18 performs a discrete cosine transformation but, it is contemplated that other transformations may be used. The transformed data signals are then quantized at block 74 by the quantization module 20. At block 76, the data signals of the currently processed image are used to generate an interpolated image which is fed back into the modified motion compensation module 52 at block 66.
The quantized data signals generated at block 74 are encoded at block 78 for further processing by the application which incorporates or uses the video coder of the present invention.
In the motion detection stage performed by the modified motion detection module 50, the operation of which is illustrated in detail in FIG. 7, the macroblocks of the currently processing image processed are classified as either moving or stationary. In order to classify the macroblocks, the modified motion detection module 50 determines if the macroblock has moved based upon predetermined motion criteria. The modified motion detector module 50 then, using the predetermined motion criteria, compares the currently processing macroblock with a macroblock which is in the same location in the previous image, the interpolated image generated by the feedback module 24, as the currently processing macroblock is in the current image.
The pixel-by-pixel absolute difference between the macroblock in the currently processing image and the macroblock at the same location in the previous image is calculated. If the difference between a pixel in the currently processing image and a pixel in the same location in the previous image is above a predetermined threshold, then that pixel is classified as moving within that macroblock. If the number of pixels classified as moving are above a second predetermined threshold, then the macroblock is classified as moving, otherwise the macroblock is classified as stationary. If the macroblock is stationary then the motion compensation phase is skipped entirely and computation proceeds as shown in FIG. 6.
Returning to FIG. 7, if, at decision block 110, the modified motion detector module 50 is processing a macroblock in the first frame of data representing the current image (i.e., if it is an INTRA coded block), that macroblock is classified as stationary at block 134 and processing by the modified motion detector module 50 terminates. If, at decision block 110, the current macroblock is not the first macroblock being processed, processing continues at block 112 where a counter and a maximum data signal indicator are initialized. Operation then continues at block 114 where the first pixel in the currently processing macroblock is retrieved.
If, at decision block 116, the signal value of the current pixel in the currently processing macroblock is greater then the current value of the maximum data signal indicator, then the value of the maximum data signal indicator is replaced with the signal value at block 118 and operation continues at block 120. If, at decision block 116, the signal value of the current pixel is not greater than the current value of the maximum data signal indicator then operation continues at block 120 where the modified motion detector module 50 determines the difference between the signal value of the current pixel and the signal value of the pixel in the same location in the previous image.
If, at decision block 122, that difference is greater than the first predetermined threshold, the counter is incremented at block 124 and operation continues at decision block 126. If, at decision block 122, the difference is not greater that the
|
