CA2284810A1

CA2284810A1 - Joint rate control for stereoscopic video coding

Info

Publication number: CA2284810A1
Application number: CA002284810A
Authority: CA
Inventors: Limin Wang
Original assignee: General Instrument Corp
Current assignee: Arris Technology Inc
Priority date: 1998-10-30
Filing date: 1999-09-30
Publication date: 2000-04-30
Also published as: EP0998151A2; EP0998151A3; US6151362A

Abstract

A method and apparatus are provided for controlling and balancing a quality ratio between left and right channels of a stereoscopic video sequence. A complexity measure for each of the channels is first determined.
Thereafter, a number of bits representing the video sequence is allocated between the channels according to each channel's complexity. The resultant joint rate control allows the bit rates for the two channels to vary according to the complexity measures of their pictures, while the aggregate bit rate is maintained at the channel capacity, which is a constant value.

Description

JOINT RATE CONTROL FOR STEREOSCOPIC VIDEO CODING
This application claims the benefit of U.S.
provisional application no. 60/106,284 filed on October 30, 1998.
S BACKGROUND OF THE INVENTION
The present invention relates to digital video signal processing techniques which may be used, for example, in connection with digital television, and more particularly to the coding of different channels of a stereoscopic video sequence.
A stereoscopic video sequence consists of two channels, which are referred to as the left and right channels. ~1'he two channels can be coded either independently or jointly. In independent coding the two channels are usually coded at the same bit rate, as there is no particular reason why one channel should be coded with fewer bits than the other channel. The same bit rate likely results in similar quality for the two channels because of the similarity between the channels. However, from the compression point of view, independent coding of the two channels of a stereoscopic sequence is not very efficient because it does not exploit the cross-correlation between two channels.
Disparity estimation as permitted by the Moving Picture Experts Group "MPEG-2 Multi-View Profile"
employs the cross-correlation between the two channels.
In the MPEG-2 Multi-View Profile, one channel (e.g. the left channel) is considered as a base layer and the second channel (e. g. the right channel) an enhancement layer. The base layer channel is coded independently while the enhancement channel is coded based upon the base layer channel. Simulations have demonstrated a better performance when cross-correlation between the left and right channels is utilized.
MPEG defines three picture types in terms of temporal processing. They are the I (intra-frame coded) picture, P (forward temporal predictive coded) picture, and B (bi-directional temporal predictive coded) picture. The input video sequence is often divided into groups of pictures (GoP). Each GoP contains one I
picture, a number of P pictures, and optionally a few B
pictures between I or P pictures. I pictures are "intraframe" coded pictures, which are not based on any other pictures (frames) in the sequence. P pictures are "predicted" pictures, which are predicted from previous I or P frames. B pictures are "bi-directionally"

predicted pictures, which can be predicted from previous and/or future I and/or P pictures.
Prior to the present invention, schemes for stereoscopic video coding typically used a fixed bit rate for each channel. In general, however, the channel (e.g., the right channel) that is coded based on the other channel (e. g., the left channel) needs fewer bits than the independently coded (e. g., left) channel for similar quality. This is because the non-independently coded (e.g., right) channel has additional information for prediction (e. g., the left channel) and it also has no I pictures which typically use many more bits than P and B pictures.
However, even where one channel is coded based on the other, the optimal sharing of a given channel capacity by the two channels still remains a question.
The problem is that the cross-correlation between two channels for different stereoscopic materials can be different, and even within a stereoscopic sequence, the degree of correlation between the two channels can also vary from time to time. Hence, a predetermined bit allocation between two channels which might be quite reasonable for some stereoscopic video sequences, or some segments of a stereoscopic sequence, could be very bad for other sequences, or segments of the same stereoscopic sequence, if equal quality between the two channels is the criterion.
Thus, it is difficult to maintain similar quality, or to balance the quality, for the two channels by using a fixed bit allocation strategy for stereoscopic video coding.
It would be advantageous to provide a method and apparatus for jointly controlling the bit rate of the two channels provided for stereoscopic video coding.
Such a scheme should allow the bit rates of the two channels to vary according to various criteria, while maintaining the aggregate rate at a constant value within the capacity of a communications channel over which the data are communicated.
The present invention provides a method and apparatus having the aforementioned and other advantages.

S
SU1~1ARY OF THE INVENTION
In accordance with the present invention, a joint rate control strategy is provided with dynamic bit allocation for stereoscopic video coding. In the joint rate control, the bit rates for the two channels are allowed to vary according to the complexity measures of their pictures, but the aggregate rate is maintained at a constant value within the channel capacity of the communication channel over which the data are carried.
A method in accordance with the invention balances a quality ratio between a plurality of channels of a stereoscopic video sequence. A
complexity measure for each of the channels is determined. Then, the number of bits representing the video sequence between the channels is allocated according to each channel's complexity. The allocating step maintains an aggregate bit rate of the video sequence at a substantially constant rate. The constant rate is determined by a channel capacity of a communications channel over which the video sequence is to be communicated.
A corresponding apparatus is also provided.

BRIEF DESCRIPTION OF THE DRAWINGS
Figure 1 provides an example of a group of pictures (GoP) structure, with arrows indicating prediction directions;
Figure 2 provides an example of picture types for both channels of a stereoscopic sequence;
Figure 3 illustrates the fullness of a decoder buffer over time under control of the present invention; and Figure 4 is a block diagram illustrating apparatus in accordance with the invention.

DETAILED DESCRIPTION OF THE INVENTION
The present invention provides a joint rate control strategy with dynamic bit allocation for stereoscopic video coding. A target bit rate is determined for a joint group of pictures (GoP). Then, the target number of bits T for the joint GoP is distributed over the frames within the joint GoP to determine a target rate for a frame. The same principle can also be applied to the slice level, wherein for a given target number of bits for a frame, a determination is made as to how many bits should be assigned to a slice of the frame. Once the target rates have been determined, rate control is effected, e.g., by adjusting a quantization parameter in the encoder. Each of these steps is discussed in greater detail below.

1. Target Rate for Joint GoP
First, the concept of a joint GoP is introduced for a stereoscopic sequence. Figure 1 shows an example of a GoP structure generally designated 10, where there are one I picture 12, four P pictures 14 and two B pictures 16 between each pair of consecutive I-P, P-P or P-I
pictures. Note that in Figure 1, the arrows 18 and 19 between the I and P pictures show the prediction directions.
A joint GoP as defined herein is actually a collection of two Goes, one from each channel of the stereoscopic sequence for the same interval. This is illustrated in Figure 2, which shows an example of picture types (I, P, B) allowed in the MPEG-2 Multi-View Profile, for both the left and right channels 20, 22 respectively, of a stereoscopic video sequence.
The left channel 20 is coded independently, and it has all three of the picture types defined in MPEG.
However, the right channel 22 is coded based upon the left channel, as indicated by the prediction directions 24, 26 in Figure 2.
As can be seen, the right channel has no I
pictures, but only P and B pictures. Furthermore, P
and B pictures in the right channel are different from P and B pictures in the left channel in the nature of prediction. The predictions in P and B pictures for the right channel involve a disparity estimation from the pictures in the left channel. Specifically, the references for P and B pictures in the right-channel can be the pictures in the left-channel. To distinguish the two types of P and B pictures, let us call the P and B pictures in the left channel the left-channel P and B pictures, and the P and B
pictures in the right-channel the right channel P and B pictures, respectively. Note that in Figure 2, the arrows 24, 26 and 28 show the prediction directions.
If the GoP length for each channel is N, a joint GoP
contains 2N frames. A stereoscopic video sequence of two channels is now divided into joint Goes. Each joint GoP has the same total number of left channel I
pictures, left-channel P and B pictures, and right-channel P and B pictures as every other joint GoP.
Hence, the same nominal number of bits can be assigned for each joint GoP, that is:
2 0 ( 1 ) T = 2NR~na""er~nPf>
where:
Rchannel(bpj) is the channel rate in bits/frame. It is noted that different embodiments are possible where, for example, the I pictures are provided in the right-channel instead of the left-channel.
2. Target Rate for a Frame Given a target number of bits, T, for a joint 5 GoP, the next step is to distribute the T bits over the frames within the joint GoP. The strategy adopted here is based upon the frame complexity. Similar to Test Model 5 (TM5), as disclosed in ISO/IEC (MPEG-2), "Test Model 5", MPEG 93/457 AVC-491, April 1993 10 and incorporated herein by reference, a complexity measure for a frame is defined in accordance with the present invention. This complexity measure comprises the product of the (average) quantization parameter (Q) used for the frame and the number of bits (R) generated for the frame by using Q, i.e..
(2) C=QR.
Hence, for a joint GoP of 2N frames, there can be 2N complexity measures, one for each of 2N frames, i.e..
2 0 ( 3 ) Cm.n - Qm,nRm,n ~ m = 1,2 and h =1,2,..., N
where m is the channel index (here, "1" is assigned for the left channel and "2" for the right channel) and n is the frame index. Let Tm." be the target number of bits assigned for frame(m,n). The actual rate for frame (m, n), Rm.", should be made close to the target rate,Tm,n, by proper rate control, i.e.:
Cm,n ( ) Tm,n = Rm,n =
~m,n Similarly, the total number of bits generated from the 2N frames in each joint GoP should also be close to the nominal target rate for the joint GoP, T, i.e..

( 5 ) T ~ ~ Rm.n m=1 n=1 If equal quality is required, the same quantization parameter should be applied to all of the frames because quantization is the only lossy operation in MPEG video coding. Considering the difference in the three picture types, as in TMS, we can weight the quantization parameter by a constant factor for the different picture types, i.e.:
( 6 ) ~m.rt = Km.n where .

K, for I picture in left channel K,P for P picture in left channel ( 7 ) Km,n = Kra for B picture in left channel K,p for P picture in right channel KrB for B picture in right channel In TM5, the constant factors for I and P pictures are equal to 1 and for B pictures the constant factor is 1.4. The quality balance between the two channels can also be controlled by introducing a weighting factor for each channel m, wm, as:
( 6 a ) ~m,n - wm Km.n where:
w~ for left channel (8) wm =
w, for right channel Equations for Tm,n and T can therefore be rewritten as ( 9 ) Tm,n = 1 Cm,n arid wm Km.n ( 10 ) T - Lr L_r Rm.n 1 m=1 n=t ~rn=1 n=1 wmKm,n From equations (9) and (10), the target number of bits for each frame(m,n)can be determined as:

C
( 11 ) Tn~ - 2 rVm~C'n T
m,n ~~ Wm Km.n Here, the numerator is the weighted complexity measure for frame (m,n) and the denominator is the sum of the weighted complexity measures for all the frames within the current joint GoP.
One concern with the above equation is that in order to determine a target number of bits for a frame, it is necessary to know the complexity measures for all of the frames of the associated joint GoP.
This may not be practical, and therefore it is desirable to modify the above equation. To modify the equation, at each frame, the first consideration is only how the remaining number of bits is distributed over the remaining frames in a joint GoP. Secondly, the assumption is made that all of the future frames of the same picture type (I, the left-channel P and B, the right-channel P and B) have the same complexity measure. This is a reasonable assumption for a continuous scene.
Thus, for each picture type, only one complexity measure has to be maintained; C, for the I picture, C,P for the left-channel P, C,B for the left-channel B, C,p for the right-channel P, and CrB for the right-channel B. The bit allocation strategy therefore becomes:
(12) Cm.n - Wm Km.n ,n 1 Cl Nl + 1 CrP NlP + 1 CIB NIB + 1 CrP NrP + 1 CrB NrB r ~'r Ku u'! KIP u'r K!B ~'r KrP ~'r KrB
where:
~ Cmn is the complexity measure for the picture type of frame (m, r.-) (Note that after encoding a frame, the complexity measure of the corresponding picture type needs to be updated according to equation (2), set forth above), ~ N,,N,P,N,B,NrP and N,B are, respectively, the number of the remaining I pictures, left-channel P pictures, left-channel B pictures, right-channel P pictures and right-channel B pictures, in the current joint GoP, ~ T, is the remaining number of bits in the current joint GoP, defined as:
( 13 ) Tr = Tr - Rm,,n where Rm,,n, is the number of bits generated for frame(m',n') -- the frame coded just before frame (m, n) .
The above equation actually indicates how the remaining number of bits is distributed over the 5 remaining frames within a joint GoP. At the beginning of each joint GoP, T, needs to be updated as follows:
(14) T,=T,+T
where T is the nominal number of bits assigned for each joint GoP. The remaining number of bits after 10 encoding the last frame in a joint GoP, T,, can be either a positive or negative number, and it is passed to the next joint GoP by equation (14).
3. Target Rate for a Slice The principles disclosed above can also be 15 applied to the slice level. For a given target number of bits, Tm,", for frame (m, n), it may be necessary to decide how many bits out of Tm,n, should be assigned to slice s of the frame. Similar to the frame complexity, a slice complexity measure, C, is defined as the product of the (average) quantization parameter Q and the number of bits R generated by using Q for the slice. If a frame contains S slices, there will be S slice complexity measures, i.e.:
( 15 ) C, = QS RS s = 1,2.,...,5 Because of the variation in spatial content of a frame, the S slice complexity measures can be quite different even within the same frame. By using the same methodology, it can be shown that for the same coding distortion, the target number of bits for each slice s of frame (m, n) should be ( 16 ) T - SCm'"'J Tm,n,.
m,n,s Cm,n.s' s'=s where T",,",, is the remaining number of bits up to slice s-1 for frame(m,n), defined as s-~
( 17 ) Tm,n,. = Tm.n _ ~ Rs, s~_~
and RS, is the number of bits generated for slice s'.
It should be pointed out that all of the Cm,",Sused in equation (1~6) can actually be calculated from the previously coded frame of the same type by using equation (15).
4. Buffer Modeling and Protection The bit allocation strategy according to equation (12) assigns a frame a target rate based upon the frame complexity measure and picture type, as well as the program GOP structure. Hence, both channels may behave more or less like variable bit rate (VBR) video even though the aggregate rate may be constant (for constant-rate channel applications). It is the responsibility of the encoder to prevent both the encoder and decoder buffers from overflowing and underflowing. For constant rate channel applications, encoder buffer modeling and protection is the same as in the constant bit rate (CBR) case. However, the decoder may only be able to decode the base layer channel (e.g. the left channel) which is VBR. Decoder buffer modeling and protection for VBR
video is a challenging issue. It is possible to achieve decoder buffer protection by applying additional constraints on the target rate of frames.
For purposes of explanation, it is assumed that the decoder can only decode the base layer channel (e.g. the left channel) of m=1. Only the packets of the selected channel will be extracted and decoded.
Further, it is assumed that the decoding delay is N' frames. The quantity Rm,n is defined as the number of bits transmitted for channel m during the nth frame period, as shown in Figure 3. The decoder buffer will therefore be filled up to ~n~o'Rm,n before any bits are moved out. This is depicted in Figure 3, where it can be seen that the quantity of data in the buffer is maintained between zero and an upper threshold 30 which corresponds to a full buffer. At the frame instant of n+N', the compressed bits for frame n, Rm,n are moved out the decoder buffer and moved into the decoder instantaneously, and between the frame instants of n+N' and n+N'+1, the decoder buffer receives Rm,"+N~ bits from the network, also shown in Figure 3.
Let Bm~+N' be the decoder buffer fullness at the frame instant of n+N', after the compressed bits for frame n are moved out ( Figure 3 ) . Clearly, B,d",n+N' is a local minimum and has to be equal to, or greater than, zero, i.e., ( 18 ) 0 <_ Bm,n+N' Furthermore, the buffer fullness right before the frame instant of n+N'+1 is BMn+N' +R,~,n+N~ ~ as shown in Figure 3. This is the local maximum and has to be equal to, or smaller than, the maximum decoder buffer size; say Bm~ , that is d c d ( 19 ) Bm +N' + R
' ~ B

,n m,n+N
max From (18) and (19), we have, ( 2 0 ) 0 ~ Bm n+N' ~ -R
c N' + B

, r n,n+
max The buffer fullness at the frame instant of n+N' , B,d",n+N ' , ( Figure 3 ) can be given as ( 2 0 a ) Bm ...
n+N' - Bm n =1 n+N'-1 + Rm n+N'-I - R

, , , , , m,n n_1 n ( 2 Ob ) R
, = Bm N' + ~ Rm n'+N' m.n , , n'=0 n'=1 N'-I n+N'-I n ) _ ~ ~ Rm n, Rm.O ) ~- ~ Rm n, - ~ Rm'n n~=0 n'=N' n'=1 n+N'-1 n-1 n-1 ( 2 0 d ) - ~ Rm n, ~' ~~ Rm n, - ~ R
, ~ R

.
, m.n m,n n'=n n'=0 n'=0 n+N'-1 1 ~ ( 2 0 a ) - ~ Rm,n' - Bm.n -Rm,n n'=n where n-1 n-1 ( 2 1 ) Bm n = ~ Rm n, - ~ Rm n, , , .

n'=0 n'=0 - can b e considered as the fullness of a virtual encoder buffe r at the frame instant of n for channel m. From ( 18 ) and ( 2 0 ) , we have n+N'-1 Rm n, Bm n Rm n ~ Rm n+N' + Bmax , , , , n'=n or n+N' n+N'-I
2 3 ) ~ Rlnrn' Bm,n Bmax ~ IZnr,n < ~ Rnr n, Bnr'n n'=n n'=n This is a constraint on the number of bits for frame n of channel m. If the target rate for a frame can be met, the constraint on rate for a frame can be 5 translated into the constraint on target rate for the frame, i.e., n+N' n+N'-1 ( 2 4 ) ~ ~ m,n' Bnr,n Bmax C ~ r,n ~ ~ Rm,ri Bm,n n'=n ri =n Hence, before starting to encode frame n of channel m, a determination is made as to whether its target rate 10 is within the proper range, and if not, the target rate is adjusted using the following equation (25):
~n+N' ne -Be -Bd n+N' c _ a _ d LIn'=n m,n' nr,n mae 1 ~ r,n C ~n'=n ~m~n' Bnr,n Bmax ~n+N'-I ~~ Be 1 ~ > n+N'-1 Re - Be . Tm'n n~=n mW mW m,n ~n'_n m,n' nr,n - Tm~n otherwise Note that Be is the fullness of a virtual encoder nr, n buffer for channel m at the frame instant of n, 15 defined in (21), and it is available. On the other hand; the transmission rates for channel m during the intervals of the future frames, Rm.n.,n'=n,n+1,...,n+N', can be pre-determined. In order to determine the transmission rate of each channel, the strategy adopted to is to make:
(a) the average input rate to the decoder buffer equal to the average output rate, and (b) the total transmission rate of the two channels equal to the channel rate.
Specifically, the transmission rate for frame n of channel m is set equal to the average number of compressed bits for the N previous frames that are most recently moved out the decoder buffer (Figure 3), i.e., 1 n~_N.
( 2 6 ) Rm,n. _ - ~ Rm.n,. n'= n, n + 1,..., n + N' N n"=n'-N'-N+1 where N is the joint GOP length. Clearly, the average input rate to the decoder buffer is equal to the average output rate. The total transmission rate of the two channels for any frame period of n, ~m=IRm,n , is also equal to the channel rate, 2R~hannel(bpf) . i . a . , 2 2 1 n.-N.
( 2 7 a ) ~ Rm ». - ~ ~ Rm,n"
m=1 m=1 N n"=n'-N'-N+1 2 n'-N' (27b) N ~ G.Rm.n°) m=1 n"=n'-N'-N+1 ( 2 7 c ) _ ~ T = 2 2~ T = 2R~,,~nne~~bp~-~
With the future transmission rates defined in (26), all the terms in the upper bound for frame n of n+N'-I
channe 1 111, ~n,_n Rm,n' - ~m,n i ~-n ( 2 3 ) and ( 2 4 ) are available at the frame instant of n, or the upper bound can be calculated based upon the information of the previous frames n-1, n-2, .... However, in the lower bound Of ~n~n'Rn,,n' -Bm,n Bronx In (23) and (24) , the term 1 (R +...+R +12 , ) requires the information Rm.n+N' ~ m,n-N+I m,n-I nt n on the number of bits for frame n, Rn,,,~, which is not available yet at the frame instant of n. By substituting (26) into the lower bound in (23), we have n+N'-I 1 n-I
( 2 8 ) ~ Rm,n' + - ( ~ Rm,n" + Rm,n ) Bm.n Bmax ~ Rm,n n'=n N n"=n-N.EI
1S Or n+N'-1 1 n-I
( 2 9 ) ~ Rm,n' + ~ Rm,n" Bm,n Bmax C (~ ~ )Rm.n n'=n N n"=n-N+I N
The lower bound for frame n of channel m can therefore be written as:

n+N'-1 1 n-~
( 3 O ) ~ - ~ ~ ~ Rm,n' + ~ ~ Rnr,n" Bnt,n Bmax ~ ~ Rnr,n - T r,n n~-n n~~=n_N.nl where all the associated terms in the lower bound are now measurable at the frame instant of n.
5. Rate Control Once target rates have been determined, the next step in accordance with the invention is to meet these target rates. This can be achieved, for example, by adjusting the quantization parameter (Q) in an MPEG
encoder. The Q can be adjusted on a macroblock-by-macroblock basis according to the channel buffer fullness, as in TMS; however, this may result in a non-uniform picture quality over a frame as discussed in L. Wang, nBit rate control for MPEG video coding", Proc. SPIE on Visual Comm. And Image Processing, pp.
53-64, May, 1995. For example, two identical blocks in a frame may be assigned different quantization parameters just because of the different buffer fullness levels at different times, and therefore are quantized to different quality. An alternative is to use a global quantization parameter for all of the macroblocks in a frame. In this case, identical blocks will be assigned the same quantization parameter and are quantized to the same quality. The global quantization parameter is selected so that the resulting rate is closest to the target rate. The rate control issue then becomes how to find such a global quantization parameter.
An iterative algorithm which moves the quantization parameter one step closer to the optimal level at each iteration is presented in L. Wang, "Bit rate control for hybrid DPCM/DCT video codec", IEEE
Trans. CSVT, vol. 4, pp. 509-517, Oct. 1994. When starting with the quantization parameter used for the previous frame of the same type, the iteration algorithm often needs only one or two steps to reach the best quantization parameter. However, it may take many steps to reach the optimal parameter for the frames at scene cuts. A binary search algorithm which guarantees the best quantization parameter in five steps is disclosed in L. Wang, "Bit rate control for MPEG video coding", Proc. SPIE on Visual Comm. And . Image Processing, pp. 53-64, May, 1995. Both the iterative and binary algorithms are, however, multi-pass processes, each process involving one quantization and one variable length coding step.
Hence, they require more operations than TMS.
However, the use of a proper global quantization parameter for a frame usually results in a better subjective picture quality than TM5.
A block diagram of apparatus for implementing the invention is provided in Figure 4. It is noted that 5 the apparatus illustrated in Figure 4 is for purposes of example only, and that other implementations are possible.
Left and right channel data are provided to a computer processor 40 and a bit allocation function 10 42. The processor 40, in cooperation with a complexity determination function 44, provides a complexity measure for each channel. The complexity measure is passed by processor 40 to the bit allocation function 42, which allocates a number of 15 bits to each of the left and right channels based on the complexity of each channel and the channel capacity of a communication channel 50 over which the data will ultimately be communicated. The bit __ allocated left and right channel data (which is 20 typically fewer bits than the original left and right channel data input to processor 40) is provided to an encoder 46. The encoder encodes the data in a conventional manner for transmission, e.g., by compressing the data and providing error correction 25 information as well known in the art. The encoded data is provided to a transmitter 48, which provides a data stream for communication to a receiver 52 over channel 50. As will be appreciated by those skilled in the art, the transmitter will modulate the data onto a suitable carrier if the data is to be communicated at other than baseband.
It should now be appreciated that the present invention provides a mechanism for controlling and balancing the quality ratio between the two channels of stereoscopic video sequences. In particular, the biis between the two channels are dynamically allocated according to their corresponding complexity.
It is noted that the complexity measure C can be any complexity measure of a frame, and is not limited to the product of Q and R as set forth in the illustrated embodiment.
Although the invention has been described herein in connection with various preferred embodiments, it should be appreciated that various modifications and adaptations may be made thereto without departing from the scope of the invention as set forth in the following claims.

Claims

1. A method for balancing a quality ratio between a plurality of channels of a stereoscopic video sequence, comprising the steps of:
determining a complexity measure for each of said channels; and allocating a number of bits representing said video sequence between said channels according to each channel's complexity.

2. A method in accordance with claim 1, wherein said allocating step maintains an aggregate bit rate of the video sequence at a substantially constant rate.

3. A method in accordance with claim 2, wherein said constant rate is determined by a channel capacity of a communications channel over which said video sequence is to be communicated.

4. A method in accordance with one of the preceding claims wherein:
said channels carry successive image frames of said stereoscopic video sequence; and the complexity measure of a channel is based on the complexity (C) of the image frames carried by the channel.

5. A method in accordance with claim 4 wherein the complexity (C) of an image frame comprises the product of an average quantization parameter (Q) used for that frame and the number of bits (R) generated when that frame is quantized using the quantization parameter Q.

6. A method in accordance with claim 5 wherein said allocating step is responsive to a target number of bits (Tm,n) assigned to each frame based on the complexity (C) of the frame.

7. A method in accordance with claim 6, comprising the further steps of:
grouping a number N of frames of each channel into a respective group of pictures (GoP) for the channel;
forming a joint GoP from corresponding Goes of first and second channels, said joint GoP comprising 2N
frames; and determining a nominal target number of bits (T) for the joint GoP;
wherein said allocating step is responsive to said nominal target number of bits (T).

8. A method in accordance with claim 6 or 7 comprising the further step of:
determining whether the target number of bits assigned to a particular frame is within a predetermined range and, if not, adjusting the target rate to be within said range prior to encoding that particular frame.

9. A method in accordance with one of claims 5 to 8 comprising the further step of independently weighting the quantization parameter (Q) for each different type of image frame and the particular channel in which the image frame is carried.

10. A method in accordance with one of claims 4 to 9 wherein:
the successive image frames of a first channel comprise I, P and B pictures; and the successive image frames of a second channel comprise only P and B pictures.

11. A method in accordance with claim 10 wherein:
a complexity measure C I is maintained for I
pictures;
a complexity measure C1P is maintained for the first channel P pictures;

a complexity measure C1B is maintained for the first channel B pictures;
a complexity measure C2P is maintained for the second channel P pictures; and a complexity measure C2B is maintained for the second channel B pictures.

12. A method in accordance with claim 11 comprising the further steps of:
grouping a number N of frames of each channel into a respective group of pictures (GoP) for the channel; and forming a joint GoP from corresponding Goes of said first and second channels, said joint GoP
comprising 2N frames;
wherein said allocating step is responsive to a target number of bits (T m,n) determined as:

where:
C m,n is the complexity measure for the picture type of frame (m,n), N1, N1P, N1B, N2P and N2B are, respectively, the number of remaining I pictures, first-channel P
pictures, first-channel B pictures, second-channel P
pictures and second-channel B pictures in the joint GoP, and Tr is the remaining number of bits in the joint GoP, defined as T~ = T~ - R m',n', where R m',n' is the number of bits generated for a frame (m',n') coded just before frame (m,n).

13. A method in accordance with one of the preceding claims wherein:
said channels carry successive image frames of said stereoscopic video sequence, each image frame being divided into a plurality of slices; and the complexity measure of a channel is based on the complexity (C) of the image frame slices carried by the channel.

14. A method in accordance with claim 13 wherein:
the complexity (C) of an image frame slice comprises the product of a quantization parameter (Q) used for that slice and the number of bits (R) generated when that slice is quantized using the quantization parameter Q.

15. A method in accordance with claim 14 wherein said allocating step is responsive to a target number of bits (T m,n,s) assigned to each slice based on the complexity (C) of the slice.

16. A method in accordance with claim 15, wherein:
each image frame is divided into S slices resulting in S slice complexity measures for each frame;
and the target number of bits (T m,n,s) for each slice s of a frame (m,n) is:
where:
T m,n,r is a remaining number of bits up to slice s-1 for frame (m,n), defined as :
and Rs' is a number of bits generated for slice

17. A method in accordance with claim 16, wherein the complexity measures C M,n.s are calculated from a previously coded frame of the same type as the current frame by multiplying the quantization parameter Q for each slice by the number of bits R in that slice.

18. A method in accordance with claim 1 wherein:
said channels carry successive image frames of said stereoscopic video sequence;
the complexity measure of a channel is based on the complexity (C) of the image frames carried by the channel; and said allocating step is responsive to a target bit rate T m,n assigned to each frame based on the complexity (C) of the frame;
said method comprising the further step of controlling said target bit rate to prevent the underflow and overflow of buffers in a video encoder processing said video image and a video decoder receiving said video image.

19. A method in accordance with claim 18, wherein said target bit rate is adjusted in accordance with variable lower and upper bounds for each frame.

20. A method in accordance with claim 19, wherein said target bit rate is maintained within said lower and upper bounds according to the parameters:
where the quantity R~,n is the number of bits transmitted for channel m during the nth frame period, B~,n is the fullness of a virtual encoder buffer for channel m at the frame instant of n, and B d max is the maximum fullness of a decoder buffer of said video decoder.

21. A method in accordance with claim 20 comprising the further step of predetermining the transmission rates for future frames of channel m to ensure that:
(i) an average input rate to the decoder buffer will be equal to an average output rate thereof, and (ii) a total transmission rate of left and right stereoscopic channels is equal to a total available channel rate.

22. A method in accordance with claim 21 wherein the transmission rate for a frame n of channel m is set equal to the average number of compressed bits for each of N previous frames.

23. A method in accordance with claim 22 wherein N is the number of frames in a joint group of pictures (GOP) .

24. A method in accordance with claim 19, wherein said target bit rate is maintained within said lower and upper bounds according to the parameters Tm,n otherwise, where the quantity N is the number of frames in a joint group of pictures (GOP), R~,n is the number of bits transmitted for channel m during the nth frame period, B~,n is the fullness of a virtual encoder buffer for channel m at the frame instant of n, and B d max is the maximum fullness of a decoder buffer of said video decoder;
such that all of the terms in the lower bound are determinable at the video encoder.

25. Apparatus for balancing a quality ratio between a plurality of channels of a stereoscopic video sequence, comprising:
a processor adapted to determine a complexity measure for each of said channels; and means for allocating a number of bits representing said video sequence between said channels according to each channel's complexity.

26. Apparatus in accordance with claim 25, wherein said allocating means maintains an aggregate bit rate of the video sequence at a substantially constant rate.

27. Apparatus in accordance with claim 26, wherein said constant rate is determined by a channel capacity of a communications channel over which said video sequence is to be communicated.