WO2007081176A1 - Processing multiview video - Google Patents

Processing multiview video Download PDF

Info

Publication number
WO2007081176A1
WO2007081176A1 PCT/KR2007/000225 KR2007000225W WO2007081176A1 WO 2007081176 A1 WO2007081176 A1 WO 2007081176A1 KR 2007000225 W KR2007000225 W KR 2007000225W WO 2007081176 A1 WO2007081176 A1 WO 2007081176A1
Authority
WO
WIPO (PCT)
Prior art keywords
block
illumination compensation
neighboring block
neighboring
value
Prior art date
Application number
PCT/KR2007/000225
Other languages
French (fr)
Inventor
Jeong Hyu Yang
Original Assignee
Lg Electronics Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from KR1020060037773A external-priority patent/KR20070076356A/en
Priority claimed from KR1020060110338A external-priority patent/KR20070076392A/en
Priority claimed from KR1020060110337A external-priority patent/KR20070076391A/en
Application filed by Lg Electronics Inc. filed Critical Lg Electronics Inc.
Priority to JP2008550241A priority Critical patent/JP5192393B2/en
Priority to CN200780003083.8A priority patent/CN101371571B/en
Priority to EP07700952A priority patent/EP1982517A4/en
Priority to KR1020087019748A priority patent/KR101276847B1/en
Publication of WO2007081176A1 publication Critical patent/WO2007081176A1/en

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/577—Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/105—Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136—Incoming video signal characteristics or properties
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146—Data rate or code amount at the encoder output
    • H04N19/147—Data rate or code amount at the encoder output according to rate distortion criteria
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/174—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a slice, e.g. a line of blocks or a group of blocks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/187—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a scalable video layer
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/189—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding
    • H04N19/19—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding using optimisation based on Lagrange multipliers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/189—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding
    • H04N19/196—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters
    • H04N19/197—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters including determination of the initial value of an encoding parameter
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/44—Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/46—Embedding additional information in the video signal during the compression process
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/597—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/44—Receiver circuitry for the reception of television signals according to analogue transmission standards
    • H04N5/455—Demodulation-circuits

Definitions

  • the invention relates to processing multiview video.
  • Multiview Video Coding relates to compression of video sequences (e.g., a sequence of images or
  • picture that are typically acquired by respective cameras.
  • the video sequences or “views” can be encoded according to a standard such as MPEG.
  • a picture in a video sequence can represent a full video frame or a field of a video frame.
  • a slice is an independently coded portion of a picture that includes some or all of the macroblocks in the picture, and a macroblock includes blocks of picture elements (or "pixels").
  • the video sequences can be encoded as a multiview video sequence according to the H.264/AVC codec technology, and many developers are conducting research into amendment of standardis to accomodate multiview video sequences.
  • the term "profile” indicates the standardization of technical components for use in the video encoding/decoding algorithms.
  • the profile is the set of technical components prescribed for decoding a bitstream of a compressed sequence, and may be considered to be a substandard.
  • the above-mentioned three profiles are a baseline profile, a main profile, and an extended profile.
  • a variety of functions for the encoder and the decoder have been defined in the H.264 standard, such that the encoder and the decoder can be compatible with the baseline profile, the main profile, and the extended profile respectively.
  • the bitstream for the H.264/AVC standard is structured according to a Video Coding Layer (VCL) for processing the moving-image coding (i.e., the sequence coding) , and a Network Abstraction Layer (NAL) associated with a subsystem capable of transmitting/storing encoded information.
  • VCL Video Coding Layer
  • NAL Network Abstraction Layer
  • the output data of the encoding process is VCL data, and is mapped into NAL units before it is transmitted or stored.
  • Each NAL unit includes a Raw Byte Sequence Payload (RBSP) corresponding to either compressed video data or header information.
  • the NAL unit includes a NAL header and a RBSP.
  • the NAL header includes flag information (e.g., nal__ref_idc) and identification (ID) information (e.g., nal_unit__type) .
  • the flag information "nal__ref_idc” indicates the presence or absence of a slice used as a reference picture of the NAL unit.
  • the ID information "nal_unit_type” indicates the type of the NAL unit.
  • the RBSP stores compressed original data. An RBSP trailing bit can be added to the last part of the RBSP, such that the length of the RBSP can be represented by a multiple of 8 bits.
  • NAL units there are a variety of the NAL units, for example, an Instantaneous Decoding Refresh (IDR) picture, a Sequence Parameter Set (SPS), a Picture Parameter Set (PPS), and Supplemental Enhancement Information (SEI), etc.
  • the standard has generally defined a target product using various profiles and levels, such that the target product can be implemented with appropriate costs.
  • the decoder satisfies a predetermined constraint at a corresponding profile and level .
  • the profile and the level are able to indicate a function or parameter of the decoder, such that they indicate which compressed images can be handled by the decoder.
  • Specific information indicating which one of multiple profiles corresponds to the bitstream can be identified by profile ID information.
  • the profile ID information "profile_idc” provides a flag for identifying a profile associated with the bitstream.
  • the H.264/AVC standard includes three profile identifiers (IDs). If the profile ID information "profile idc” is set to “66”, the bitstream is based on the baseline profile. If the profile ID information “profile__idc” is set to "77”, the bitstream is based on the main profile. If the profile ID information "profile__idc” is set to "88”, the bitstream is based on the extended profile.
  • the above-mentioned "profile_idc” information may be contained in the SPS (Sequence Parameter Set), for example. Disclosure of Invention
  • a method for decoding a multiview video signal comprises: receiving a bitstream comprising encodings of multiple views of the multiview video signal, each view comprising multiple pictures segmented into multiple segments (e.g., an image block segment such as a single block or a macroblock, or a segment such as a slice of an image) ; extracting flag information associated with a portion of the multiview video signal from the bitstream indicating whether illumination compensation of segments within said portion of the multiview video signal is enabled; and for a portion in which illumination compensation is enabled according to the extracted flag information, extracting from the bitstream a value associated with a segment within the portion and determining from said extracted value whether illumination compensation of the segment is to be performed, Aspects can include one or more of the following features .
  • the segments comprise image blocks.
  • the method further comprises, for a first block associated with a value that indicates that illumination compensation is to be performed, obtaining a predictor for performing illumination compensation of the first block using an offset value for illumination compensation of at least one neighboring block adjacent to the first block.
  • An offset value for illumination compensation of a neighboring block is obtained by forming a sum that includes a predictor for illumination compensation of the neighboring block and a residual value.
  • Obtaining a predictor for illumination compensation of the first block using an offset value for illumination compensation of at least one neighboring block adjacent to the first block includes selecting the at least one neighboring block according to a predetermined order among the neighboring blocks. Selecting the at least one neighboring block according to the predetermined order comprises determining whether one or more conditions are satisfied for a neighboring block in an order in which one or more vertical or horizontal neighbors are followed by one or more diagonal neighbors.
  • the flag information enables illumination compensation for one or more of a sequence, a view, a group of pictures, a picture, and a slice that contains the block.
  • the flag information enables illumination compensation for the slice that contains the block.
  • the extracted value comprises flag information for a macroblock that contains the block or flag information for the block.
  • the extracted value comprises flag information for the macroblock that contains the block.
  • a method for decoding a multiview video signal comprises: receiving a bitstream comprising encodings of multiple views of the multiview video signal, each view comprising multiple pictures segmented into multiple segments; and obtaining a predictor for illumination compensation of a first segment using an offset value for illumination compensation of at least one neighboring segment adjacent to the first segment, including selecting the at least one neighboring segment according to a predetermined order among the neighboring segments .
  • Aspects can include one or more of the following features .
  • the first segment and the at least one neighboring segment comprise image blocks.
  • An offset value for illumination compensation of a neighboring block is obtained by forming a sum that includes a predictor for illumination compensation of the neighboring block and a residual value.
  • Selecting the at least one neighboring block according to the predetermined order comprises determining whether one or more conditions are satisfied for a neighboring block in an order in which one or more vertical or horizontal neighbors are followed by one or more diagonal neighbors. Selecting the at least one neighboring block according to the predetermined order comprises determining whether one or more conditions are satisfied for a neighboring block in the order of: a left neighboring block, followed by an upper neighboring block, followed by a right-upper neighboring block, followed by a left-upper neighboring block.
  • Selecting the at least one neighboring block according to the predetermined order comprises determining whether one or more conditions are satisfied for a neighboring block in the order of: a upper neighboring block, followed by a left neighboring block, followed by a right-upper neighboring block, followed by a left-upper neighboring block.
  • Determining whether one or more conditions are satisfied for a neighboring block comprises extracting a value associated with the neighboring block from the bitstream indicating whether illumination compensation of the neighboring block is to be performed.
  • the extracted value comprises flag information for a macroblock that contains the block or flag information for the block.
  • Obtaining the predictor comprises determining whether to use an offset value for illumination compensation of a single neighboring block or multiple offset values for illumination compensation of respective neighboring blocks .
  • the method further comprises, when multiple offset values are to be used, obtaining the predictor for performing illumination compensation of the first block by combining the multiple offset values.
  • a method for decoding a multiview video signal comprises: receiving a bitstream comprising encodings of multiple views of the multiview video signal, each view comprising multiple pictures segmented into multiple segments; obtaining an offset value for illumination compensation of a first segment with respect to a reference picture, wherein the offset value is predicted using an offset value for illumination compensation of at least one neighboring segment determined based on characteristics associated with the neighboring segment; and decoding the bitstream using illumination compensation for the first segment including forming a sum that includes a predictor for pixels of the first segment obtained from the reference picture, a residual for pixels of the first segment, and a corresponding offset value for illumination compensation.
  • aspects can include one or more of the following features .
  • the first segment and the at least one neighboring segment comprise image blocks.
  • An offset value for illumination compensation of a neighboring block is obtained by forming a sum that includes a predictor for illumination compensation of the neighboring block and a residual value.
  • the method further comprises selecting at least one neighboring block based on whether one or more conditions are satisfied for a neighboring block in an order in which one or more vertical or horizontal neighbors are followed by one or more diagonal neighbors .
  • Selecting at least one neighboring block comprises determining whether one or more conditions are satisfied for a neighboring block in the order of: a left neighboring block, followed by an upper neighboring block, followed by a right-upper neighboring block, followed by a left-upper neighboring block.
  • Determining whether one or more conditions are satisfied for a neighboring block comprises extracting a value associated with the neighboring block from the bitstream indicating whether illumination compensation of the neighboring block is to be performed.
  • the extracted value comprises flag information for a macroblock that contains the block or flag information for the block.
  • Selecting at least one neighboring block comprises determining whether to use an offset value for illumination compensation of a single neighboring block or multiple offset values for illumination compensation of respective neighboring blocks.
  • the method further comprises, when multiple offset values are to be used, obtaining the predictor for performing illumination compensation of the first block by combining the multiple offset values .
  • Combining the multiple offset values comprises taking an average or median of the offset values.
  • a method for decoding a multiview video signal comprises: receiving a bitstream comprising encodings of multiple views of the multiview video signal, each view comprising multiple pictures segmented into multiple segments; obtaining a predictor for illumination compensation of a first segment with respect to a reference picture; determining an offset value for illumination compensation of the first segment including forming a sum that includes the predictor and a residual value; and decoding the bitstream using illumination compensation for the first segment including forming a sum that includes a predictor for pixels of the first segment obtained from the reference picture, a residual for pixels of the first segment, and a corresponding offset value for illumination compensation.
  • aspects can include one or more of the following features .
  • the segments comprise image blocks.
  • Using illumination compensation for the first segment comprises obtaining an offset value for illumination compensation of a neighboring block by forming a sum that includes a predictor for illumination compensation of the neighboring block and a residual value.
  • the method further comprises selecting at least one neighboring block based on whether one or more conditions are satisfied for a neighboring block in an order in which one or more vertical or horizontal neighbors are followed by one or more diagonal neighbors.
  • Selecting at least one neighboring block comprises determining whether one or more conditions are satisfied for a neighboring block in the order of: a left neighboring block, followed by , an upper neighboring block, followed by a right-upper neighboring block, followed by a left-upper neighboring block.
  • Determining whether one or more conditions are satisfied for a neighboring block comprises extracting a value associated with the neighboring block from the bitstream indicating whether illumination compensation of the neighboring block is to be performed.
  • the extracted value comprises flag information for a macroblock that contains the block or flag information for the block.
  • Selecting at least one neighboring block comprises determining whether to use an offset value for illumination compensation of a single neighboring block or multiple offset values for illumination compensation of respective neighboring blocks.
  • the method further comprises, when multiple offset values are to be used, obtaining the predictor for performing illumination compensation of the first block by combining the multiple offset values.
  • Combining the multiple offset values comprises taking an average or median of the offset values.
  • a method for decoding a multiview video signal comprises: receiving a bitstream comprising encodings of multiple views of the multiview video signal, each view comprising multiple pictures segmented into multiple segments; and obtaining a predictor for illumination compensation of a first segment with respect to a reference picture using an offset value for illumination compensation of at least one neighboring segment adjacent to the first segment according whether the reference picture associated with the first segment is the same as a reference picture associated with the neighboring segment .
  • Aspects can include one or more of the following features .
  • the segments comprise image blocks.
  • Using illumination compensation for the first segment comprises obtaining an offset value for illumination compensation of a neighboring block by forming a sum that includes a predictor for illumination compensation of the neighboring block and a residual value.
  • the method further comprises selecting at least one neighboring block based on whether one or more conditions are satisfied for a neighboring block in an order in which one or more vertical or horizontal neighbors are followed by one or more diagonal neighbors.
  • Selecting at least one neighboring block comprises determining whether to use an offset value for illumination compensation of a single neighboring block or multiple offset values for illumination compensation of respective neighboring blocks.
  • the method further comprises, when multiple offset values are to be used, obtaining the predictor for performing illumination compensation of the first block by combining the multiple offset values .
  • Combining the multiple offset values comprises taking an average or median of the offset values.
  • a method for encoding a video signal comprises generating a bitstream capable of being decoded into the video signal by the respective decoding method.
  • a method for encoding a bitstream comprises: forming the bitstream comprising encodings of multiple views of the multiview video signal, each view comprising multiple pictures segmented into multiple segments; providing flag information associated with a portion of the multiview video signal in the bitstream indicating whether illumination compensation of segments within said portion of the multiview video signal is enabled; and for a portion in which illumination compensation is enabled according to the extracted flag information, providing in the bitstream a value associated with a segment within the portion and determining from said extracted value whether illumination compensation of the segment is to be performed.
  • a method for encoding a bitstream comprises: forming the bitstream comprising encodings of multiple views of the multiview video signal, each view comprising multiple pictures segmented into multiple segments; and providing a predictor for illumination compensation of a first segment using an offset value for illumination compensation of at least one neighboring segment adjacent to the first segment, including selecting the at least one neighboring segment according to a predetermined order among the neighboring segments .
  • a method for encoding a bitstream comprises: forming the bitstream comprising encodings of multiple views of the multiview video signal, each view comprising multiple pictures segmented into multiple segments; providing an offset value for illumination compensation of a first segment with respect to a reference picture, wherein the offset value is able to be predicted using an offset value for illumination compensation of at least one neighboring segment determined based on characteristics associated with the neighboring segment; and providing information for illumination compensation for the first segment based on a sum that, includes a predictor for pixels of the first segment obtained from the reference picture, a residual for pixels of the first segment, and a corresponding offset value for illumination compensation.
  • a method for encoding a bitstream comprises: forming the bitstream comprising encodings of multiple views of the multiview video signal, each view comprising multiple pictures segmented into multiple segments; providing a predictor for illumination compensation of a first segment with respect to a reference picture; providing an offset value for illumination compensation of the first segment based on a sum that includes the predictor and a residual value; and providing information for illumination compensation for the first segment based on a sum that includes a predictor for pixels of the first segment obtained from the reference picture, a residual for pixels of the first segment, and a corresponding offset value for illumination compensation.
  • a method for encoding a bitstream comprises: forming the bitstream comprising encodings of multiple views of the multiview video signal, each view comprising multiple pictures segmented into multiple segments; and providing a predictor for illumination compensation of a first segment with respect to a reference picture using an offset value for illumination compensation of at least one neighboring segment adjacent to the first segment according whether the reference picture associated with the first segment is the same as a reference picture associated with the neighboring segment .
  • a computer program stored on a computer- readable medium, comprises instructions for causing a computer to perform the respective decoding method.
  • image data embodied on a machine-readable information carrier is capable of being decoded into a video signal by the respective decoding method.
  • a decoder comprises means for performing the respective decoding method.
  • an encoder comprises means for generating a bitstream capable of being decoded into a video signal by the respective decoding method.
  • a method for encoding a video sequence comprises: a) obtaining an average pixel value of at least one block from among neighboring blocks of a current block and reference blocks of another view; b) deriving a predicted average pixel value of the current block from the obtained average pixel value of the at least one block; and c) obtaining a difference value between a predicted average pixel value of the current block and an average pixel value of the current block.
  • a method for decoding a video sequence comprising: 1) obtaining a difference value capable of reconstructing an average pixel value of a current block from a video signal; m) deriving a predicted average pixel value of the current block from reference blocks of another view; and n) reconstructing the average pixel value of the current block on the basis of the predicted average pixel value and the difference value.
  • an apparatus for encoding a video sequence comprising: an average pixel value obtaining unit for obtaining average pixel values of neighboring blocks of a current block and reference blocks of another view; an average pixel value prediction unit for deriving a predicted average pixel value of the current block from the obtained average pixel value; and a differntial-value encoding unit for obtaining a difference value between the predicted average pixel value and the average pixel value of the current block.
  • an apparatus for decoding a video sequence comprising: a difference-value decoding unit for obtaining a difference value from a received bitstream; an average pixel value prediction unit for deriving a predicted average pixel value of a current block from a reference block of another view; and an illumination compensation unit for reconstructing the average pixel value of the current block on the basis of the predicted average pixel value and the difference value.
  • a method for decoding a video signal comprises: obtaining a predictor for performing illumination compensation of a current block using an offset value of at least one neighboring block adjacent to the current block; and reconstructing an offset value of the current block using the predictor, wherein the predictor is determined by determining whether a reference index of the current block is equal to a reference index of the neighboring block.
  • a method for decoding a video signal comprising: reconstructing a current block offset value indicating a difference between an average pixel value of a current block and an average pixel value of at least one reference block; and obtaining respectively an offset values of the reference blocks of the current block using the offset value, if the current block is predictively encoded by two or more reference blocks .
  • a method for decoding a video signal comprises: obtaining flag information indicating whether illumination compensation of a current block is performed; and if the illumination compensation is performed by the flag information, reconstructing an offset value indicating a difference between an average pixel value of the current block and an average pixel value of a reference block.
  • a method for decoding a video signal comprising: a) obtaining flag information for allowing a specific level of a video signal to be illumination-compensated; and b) decoding a specific level of the video signal illumination- compensated by the flag information, wherein the specific level of the video signal corresponds to any one of a sequence level, a view level, a GOP (Group of Pictures) level, a picture level, a slice level, a macroblock level, and a block level.
  • a method for encoding a video signal comprising: obtaining a current-block' s offset value indicating a difference between an average pixel value of the current block and a reference block; and searching for a reference optimally matched with the current block using the offset value; and obtaining a motion vector from the matched reference block, and encoding the motion vector.
  • the method or apparatus for encoding/decoding a video sequence predicts an average value of a current block to be encoded on the basis of peripheral blocks, and transmits a difference value between the current block and the peripheral blocks, thereby minimizing an amount of information to be transmitted for illumination compensation.
  • the method effectively performs illumination compensation of a multiview video sequence requiring a large amount of data, thereby increasing an encoding rate.
  • the method implements an effective encoding/decoding system using correlation between blocks or views.
  • View sequences of the multiview video data are captured by different cameras, such that there is a difference in illumination due to inner or outer factors of the cameras.
  • the method predicts an offset value of a current block using information of the neighboring block, transmits only a residual value between the current block and the neighboring block, such that it can minimize an amount of information to be transmitted for illumination compensation.
  • the method determines whether a reference index of the current block is equal to that of the neighboring block, resulting in the implementation of correct prediction.
  • the method predicts flag information indicating whether the illumination compensation of the current block is performed, and transmits only a residual value between the flag information, thereby minimizing an amount of information to be transmitted.
  • the method determines whether a reference index of the current block is equal to that of the neighboring block, resulting in the implementation of correct prediction.
  • the method uses correlation between blocks or views, resulting in the implementation of the effective coding process.
  • View sequences of the multiview video data are captured by different cameras, such that there is a difference in illumination due to inner or outer factors of the cameras.
  • the method predicts an offset value of a current block using information of the neighboring block, transmits only a residual value between the current block and the neighboring block, such that it can minimize an amount of information to be transmitted for illumination compensation.
  • the method predicts flag information indicating whether the illumination compensation of the current block is performed, and transmits only a residual value, thereby minimizing an amount of information to be transmitted. If the predictive coding process is performed using two or more reference blocks, the method employs the offset value and the flag information using at least one method, resulting in the implementation of the effective coding process. A flag bit indicating whether the illumination compensation is performed to each area of the video signal is assigned, such that the illumination compensation technique can be effectively used. The method calculates the costs by reflecting an illumination difference in the motion estimation process, resulting in the implementation of the correct predictive coding.
  • FIG. 1 is an exemplary decoding apparatus.
  • FIG. 2 is a flowchart illustrating a method for encoding a video sequence.
  • FIG. 3 is a block diagram illustrating a process for deriving a predicted average pixel value of a current block from reference blocks of other views.
  • FIG. 4 is a detailed block diagram illustrating a process for deriving a predicted average pixel value of a current block from reference blocks of other views.
  • FIG. 5 is a diagram illustrating a 16x16 macroblock.
  • FIGS. 6A-6B are diagrams illustrating 16x8 macroblocks .
  • FIGS. 7A-7B are diagrams illustrating 8x16 macroblocks .
  • FIGS. 8A-8B are diagrams illustrating 8x8 macroblocks .
  • FIG. 9 is a diagram illustrating a process for obtaining an offset value of a current block.
  • FIG. 10 is a flowchart illustrating a process for performing illumination compensation of a current block.
  • FIG. 11 is a flowchart illustrating a method for obtaining a predictor by determining whether a reference index of a current block is equal to a reference index of a neighboring block.
  • FIG. 12 is a flow chart illustrating a method for performing for an illumination compensation on the basis of a prediction type of a current block.
  • FIG. 13 is a flow chart illustrating a method for performing illumination compensation using flag information indicating whether the illumination compensation of a block is performed.
  • FIG. 14 is a flow chart illustrating a method for predicting flag information of a current block by determining whether a reference index of the current block is equal to a reference index of a neighboring block.
  • FIG. 15 is a flow chart illustrating a method for performing illumination compensation when a current block is predictively coded by two or more reference blocks.
  • FIG. 16 is a flow chart illustrating a method for performing illumination compensation using not only a flag indicating whether illumination compensation of a current block is performed, but also an offset value of a current block.
  • FIGS. 17A-17B are diagrams illustrating a method for performing illumination compensation using a flag and an offset value in association with blocks of P and B slices.
  • FIG. 18 is a flow chart illustrating a method for performing illumination compensation when a current block is predictively encoded by two or more reference blocks.
  • FIG. 19 is a flow chart illustrating a method for performing illumination compensation using a flag indicating whether illumination compensation of a current block is performed.
  • FIGS. 20A-20C are diagrams illustrating the scope of flag information indicating whether illumination compensation of a current block is performed.
  • FIG. 21 is a flow chart illustrating a method for obtaining a motion vector considering an offset value of a current block.
  • an input bitstream includes information that allows a decoding apparatus to determine whether the input bitstream relates to a multiview profile.
  • supplementary information associated with the multiview sequence is added according to a syntax to the bitstream and transmitted to the decoder.
  • the multiview profile ID can indicate a profile mode for handling multiview video data as according to an amendment of the H.264/AVC standard.
  • the MVC (Multiview Video Coding) technology is an amendment technology of the H.264/AVC standards. That is, a specific syntax is added as supplementary information for an MVC mode. Such amendment to support MVC technology can be more effective than an alternative in which an unconditional syntax is used. For example, if the profile identifier of the AVC technology is indicative of a multiview profile, the addition of multiview sequence information may increase a coding efficiency.
  • the sequence parameter set (SPS) of the H.264/AVC bitstream is indicative of header information including information (e.g., a profile, and a level) associated with the entire-sequence encoding.
  • the entire compressed moving images (i.e., a sequence) can begin at a sequence header, such that a sequence parameter set (SPS) corresponding to the header information arrives at the decoder earlier than data referred to by the parameter set.
  • the sequence parameter set RBSP acts as header information of a compressed data of moving images at entry Sl (FIG. 2) .
  • the profile ID information "profile_idc” identifies which one of profiles from among several profiles corresponds to the received bitstream.
  • the profile ID information "profile_idc” can be set, for example, to "MULTI_VIEW_PROFILE) ", so that the syntax including the profile ID information can determine whether the received bitstream relates to a multiview profile.
  • the following configuration information can be added when the received bitstream relates to the multiview profile.
  • FIG. 1 is a block diagram illustrating an exemplary decoding apparatus (or "decoder") of a multiview video system for decoding a video signalcontaining a multiview video sequence.
  • the multiview video system includes a corresponding encoding apparatus (or “encoder”) to provide the multiview video sequence as a bitstream that includes encoded image data embodied on a machine-readable information carrier (e.g., a machine-readable storage medium, or a machine-readable energy signal propagating between a transmitter and receiver. )
  • a machine-readable information carrier e.g., a machine-readable storage medium, or a machine-readable energy signal propagating between a transmitter and receiver.
  • the decoding apparatus includes a parsing unit 10, an entropy decoding unit 11, an Inverse
  • Quantization/Inverse Transform unit 12 an inter-prediction unit 13, an intra-prediction unit 14, a deblocking filter
  • the inter-prediction unit 13 includes a motion compensation unit 17, an illumination compensation unit 18, and an illumination-compensation offset prediction unit 19.
  • the parsing unit 10 performs a parsing of the received video sequence in NAL units to decode the received video sequence.
  • one or more sequence parameter sets and picture parameter sets are transmitted to a decoder before a slice header and slice data are decoded.
  • the NAL header or an extended area of the NAL header may include a variety of configuration information, for example, temporal level information, view level information, anchor picture ID information, and view ID information, etc.
  • time level information is indicative of hierarchical-structure information for providing temporal scalability from a video signal, such that sequences of a variety of time zones can be provided to a user via the above-mentioned temporal level information.
  • view level information is indicative of hierarchical-structure information for providing view scalability from the video signal.
  • the multiview video sequence can define the temporal level and view level, such that a variety of temporal sequences and view sequences can be provided to the user according to the defined temporal level and view level.
  • the user may employ the temporal scalability and the view scalability. Therefore, the user can view a sequence corresponding to a desired time and view, or can view a sequence corresponding to another limitation.
  • the above-mentioned level information may also be established in various ways according to reference conditions. For example, the level information may be changed according to a camera location, and may also be changed according to a camera arrangement type. In addition, the level information may also be arbitrarily established without a special reference.
  • anchor picture is indicative of an encoded picture in which all slices refer to only slices in a current view and not slices in other views.
  • a random access between views can be based on anchor pictures for multiview-sequence decoding.
  • Anchor picture ID information can be used to perform the random access process to access data of a specific view without requiring a large amount of data to be decoded.
  • view ID information is indicative of specific information for discriminating between a picture of a current view and a picture of another view.
  • a Picture Order Count (POC) and frame number information (frame_num) can be used.
  • inter-view prediction can be performed.
  • An identifier is used to discriminate a picture of the current view from a picture of another view.
  • a view identifier can be defined to indicate a picture's view.
  • the decoding apparatus can obtain information of a picture in a view different from a view of the current picture using the above-mentioned view identifier, such that it can decode the video signal using the information of the picture.
  • the above-mentioned view identifier can be applied to the overall encoding/decoding process of the video signal. Also, the above-mentioned view identifier can also be applied to the multiview video coding process using the frame number information "frame_num" considering a view.
  • the multiview sequence has a large amount of data, and a hierarchical encoding function of each view
  • view scalability (also called a "view scalability”) can be used for processing the large amount of data.
  • a prediction structure considering views of the multiview sequence may be defined.
  • the above-mentioned prediction structure may be defined by structuralizing the prediction order or direction of several view sequences. For example, if several view sequences to be encoded are given, a center location of the overall arrangement is set to a base view, such that view sequences to be encoded can be hierarchically selected. The end of the overall arrangement or other parts may be set to the base view.
  • the number of camera views is denoted by an exponential power of "2"
  • a hierarchical prediction structure between several view sequences may be formed on the basis of the above-mentioned case of the camera views denoted by the exponential power of "2”. Otherwise, if the number of camera views is not denoted by the exponential power of "2", virtual views can be used, and the prediction structure may be formed on the basis of the virtual views. If the camera arrangement is indicative of a two- dimensional arrangement, the prediction order may be established by turns in a horizontal or vertical direction.
  • a parsed bitstream is entropy-decoded by an entropy decoding unit 11, and data such as a coefficient of each macroblock, a motion vector, etc., are extracted.
  • the inverse quantization/inverse transform unit 12 multiplies a received quantization value by a predetermined constant to acquire a transformed coefficient value, and performs an inverse transform of the acquired coefficient value, such that it reconstructs a pixel value.
  • the inter-prediction unit 13 performs an inter-prediction function from decoded samples of the current picture using the reconstructed pixel value.
  • the deblocking filter 15 is ⁇ applied to each decoded macroblock to reduce the degree of block distortion.
  • the deblocking filter 15 performs a smoothing of the block edge, such that it improves an image quality of the decoded frame.
  • the selection of a filtering process is dependent on a boundary strength and a gradient of image samples arranged in the vicinity of the boundary.
  • the filtered pictures are stored in the decoded picture buffer 16, such that they can be outputted or be used as reference pictures.
  • the decoded picture buffer 16 stores or outputs pre- coded pictures to perform the inter-prediction function.
  • frame number information "frame_num” and POC (Picture Order Count) information of the pictures are used to store or output the pre-coded pictures .
  • Pictures of other view may exist in the above-mentioned pre-coded pictures in the case of the MVC technology. Therefore, in order to use the above-mentioned pictures as reference pictures, not only the "frarae_num" and POC information, but also view identifier indicating a picture view may be used as necessary.
  • the inter-prediction unit 13 performs the inter- prediction using the reference pictures stored in the decoded picture buffer 16.
  • the inter-coded macroblock may be divided into macroblock partitions . Each macroblock partition can be predicted by one or two reference pictures.
  • the motion compensation unit 17 compensates for a motion of the current block using the information received from the entropy decoding unit 11.
  • the motion compensation unit 17 extracts motion vectors of neighboring blocks of the current block from the video signal, and obtains a motion-vector predictor of the current block.
  • the motion compensation unit 17 compensates for the motion of the current block using a difference value between the motion vector and a predictor extracted from the video signal and the obtained motion-vector predictor.
  • the above-mentioned motion compensation may be performed by only one reference picture, or may also be performed by a plurality of reference pictures . Therefore, if the above-mentioned reference pictures are determined to be pictures of other views different from the current view, the motion compensation may be performed according to a view identifier indicating the other views.
  • a direct mode is indicative of a coding mode for predicting motion information of the current block on the basis of the motion information of a block which is completely decoded.
  • the above-mentioned direct mode can reduce the number of bits required for encoding the motion information, resulting in the increased compression efficiency.
  • a temporal direct mode predicts motion information of the current block using a correlation of motion information of a temporal direction. Similar to the temporal direct mode, the decoder can predict the motion information of the current block using a correlation of motion information of a view direction.
  • view sequences may be captured by different cameras respectively, such that a difference in illumination may occur due to internal or external factors of the cameras.
  • an illumination compensation unit 18 performs an illumination compensation function.
  • flag information may be used to indicate whether an illumination compensation at a specific level of a video signal is performed.
  • the illumination compensation unit 18 may perform the illumination compensation function using flag information indicating whether the illumination compensation of a corresponding slice or macroblock is performed.
  • the above- mentioned method for performing the illumination compensation using the above-mentioned flag information may be applied to a variety of macroblock types (e.g., an inter 16x16 mode, a B-skip mode, a direct mode, etc.)
  • information of a neighboring block or information of a block in views different from a view of the current block may be used, and an offset value of the current block may also be used.
  • the offset value of the current block is indicative of a difference value between an average pixel value of the current block and an average pixel value of a reference block corresponding to the current block.
  • a predictor of the current-block offset value may be obtained by using the neighboring blocks of the current block, and a residual value between the offset value and the predictor may be used. Therefore, the decoder can reconstruct the offset value of the current block using the residual value and the predictor. In order to obtain the predictor of the current block, information of the neighboring blocks may be used as necessary.
  • the offset value of the current block can be predicted by using the offset value of a neighboring block. Prior to predicting the current-block offset value, it is determined whether the reference index of the current block is equal to a reference index of the neighboring blocks. According to the determined result, the illumination compensation unit 18 can determine which one of neighboring blocks will be used or which value will be used.
  • the illumination compensation unit 18 may perform the illumination compensation using a prediction type of the current block. If the current block is predictively encoded by two reference blocks, the illumination compensation unit 18 may obtain an offset value corresponding to each reference block using the offset value of the current block. As described above, the inter-predicted pictures or intra-predicted pictures acquired by the illumination compensation and motion compensation are selected according to a prediction mode, and reconstructs the current picture.
  • FIG. 2 is a flow chart illustrating a method for encoding a video sequence.
  • an example of a video-sequence encoding method obtains an average pixel value of at least one block from among neighboring blocks of a current block and reference blocks of another view at step S131.
  • the video-sequence encoding method derives a predicted average pixel value of the current block using at least one mode from among several modes at step S132.
  • the video-sequence encoding method obtains a difference value between the predicted average pixel value and the actual average pixel value of the current block at step S133.
  • the video-sequence encoding method measures individual encoding efficiency of the above-mentioned several modes, and selects an optimum mode from among the several modes at step S134.
  • the above- mentioned optimum mode can be selected in various ways, for example, a method for selecting a minimum difference value from among the obtained difference values, and a method for using an equation indicating the relationship of Rate- Distortion (RD), etc.
  • the above-mentioned RD equation recognizes not only the number of encoding bits generated during the encoding of a corresponding block but also a distortion value indicating a difference value associated with an actual image, such that it calculates costs using the number of encoding bits and the distortion value.
  • the video-sequence encoding method multiplies the bit number by a Lagrange multiplier determined by a quantization coefficient, and adds the distortion value to the multiplied result, such that it calculates the costs. If the optimum mode is selected, the video-sequence encoding method can encode identification (ID) information indicating the selected mode, and transmit the encoded result. Alternatively, if the optimum mode is selected, the video-sequence encoding method can encode not only the ID information indicating the selected mode but also the difference value obtained by the selected mode, and transmit the encoded result at step S135.
  • ID identification
  • FIG. 3 is a block diagram illustrating a process for deriving a predicted average pixel value of a current block from reference blocks of another view.
  • an average pixel value of the B c block is m c
  • an average pixel value of the B r ,i block is m r ,i
  • an average pixel value of the remaining blocks is represented by the above-mentioned block notation.
  • the reference frame #1 is used as a candidate reference frame in the case of encoding the B c block.
  • a first method for predicting m c information according to information of one or more neighboring blocks is a first mode method (Model) for predicting the m c information on the basis of an average pixel value of a reference block of another view corresponding to the current block.
  • the first mode method (Model) is indicative of the method for predicting the m c information using the average pixel value the B r ,i block of the reference frame #1.
  • the difference value can be represented by the following equation 1: [Equation 1]
  • a second method for predicting a difference value between an average pixel value of a current block and an average pixel value of a reference block of another view corresponding to the current block is a second mode method (Mode2) for predicting the difference value on the basis of a difference between average pixel values of each neighboring blocks of the current block and the reference block.
  • the second mode method (Mode2) predicts a difference value between an average pixel value of the current block and an average pixel value of the B r/1 block of the reference frame #1 using a difference value in
  • the difference value can be represented by the following equation 2:
  • a third method for predicting a difference value between an average pixel value of a current block and an average pixel value of a reference block of another view corresponding to the current block is a third mode method
  • (Mode3) predicts the m c information on the basis of a difference between an average pixel value of the
  • the difference value can be represented by the following equation 3 :
  • pixel value of the current block ( B. ) and a reference block ( B 1 . , ) corresponding to the current block can be
  • FIG. 4 is a detailed block diagram illustrating a process for deriving a predicted average pixel value of a current block from reference blocks of other views.
  • FIG. 4 shows a current block, pre- encoded blocks, each of which shares a boundary with the current block, and other blocks, each of which shares a boundary with the reference block.
  • the Mode2-method equation, the Mode3-method equation, and the Mode4-method equation can be represented by the following equation 5 :
  • Equation 5 W 1 indicates a weighted coefficient.
  • the neighboring blocks used for prediction are not limited to blocks sharing a boundary, and may also include other blocks adjacent to the above-mentioned neighboring blocks as necessary. Otherwise, the above-mentioned neighboring blocks may also employ only some parts of the other blocks. The scope of the above-mentioned neighboring blocks may be
  • the reference frames of the above-mentioned Model, Mode2, Mode3, and Mode4 methods are determined to be optimum frames in consideration of rate and distortion factors after calculating several steps to an actual bitstream stage.
  • methods for selecting the optimum mode for example, a method for selecting a specific mode of a minimum difference value from among the obtained difference values, and a method for using the RD relationship.
  • the above-mentioned RD-relationship method calculates actual bitstreams of individual modes, and selects an optimum mode in consideration of the rate and the distortion.
  • the above-mentioned RD-relationship method deducts an average pixel value of each block from the current block, deducts the average pixel value of each block from the reference block, and calculates a difference value between the deducted results of the current and reference blocks, as represented by the following equation 6: [ Equation 6 ]
  • Equation 6 ⁇ x ⁇ y is indicative of a disparity
  • m r is indicative of an
  • the encoding unit has the same m r as
  • the reference block is searched for in a time domain, and an optimum block is searched for in a space-time domain. Therefore, ID information indicating whether an illumination compensation will be used is set to "0" or "1" in association with individual frames and blocks, and the resultant ID information is entropy-encoded.
  • the optimum mode it is possible to encode only the selected mode, such that the encoded result of the selected mode may be transmitted to the decoding unit.
  • a difference value obtained by the selected mode can also be encoded and transmitted.
  • the selected mode information is represented by index types, and can also be predicted by neighboring-mode information.
  • a difference value between the index of the currently- selected mode and the index of the predicted mode can also be encoded and transmitted.
  • All of the above-mentioned modes may be considered, some of the above-mentioned modes may be selected, or only one of the above-mentioned modes may also be selected as necessary. In the case of using a single method from among all available methods, there is no need to separately encode the mode index.
  • pre-decoded pixel values may be applied to current blocks of a reference frame and a target frame to be encoded.
  • pre-decoded values of left-side pixels and pre-decoded values of upper-side pixels are used to predict an average pixel value of the current block.
  • the video sequence is encoded on the basis of a macroblock.
  • the 16x16 macroblock is divided into 16x8 blocks, 8x16 blocks, and 8x8 blocks, and is then decoded.
  • the 8x8 blocks may also be divided into 8x4 blocks, 4x8 blocks, and 4x4 blocks.
  • FIG. 5 is a conceptual diagram illustrating a 16x16 macroblock for explaining usages of pre-decoded pixel values located at left- and upper- parts of an entire block in the case of deriving an average pixel value and a predicted average pixel value of a current block.
  • the 16x16 macroblock can use all the pixel values of the left- and upper- parts. Therefore, in the case of predicting an average pixel value of the current block, an average pixel value of pixels
  • FIG. 6A is a conceptual diagram illustrating a 16x8 macroblock for explaining usages of all the pixels enclosing divided blocks in the case of deriving an average pixel value and a predicted average pixel value of the divided blocks.
  • FIG. 6B is a conceptual diagram illustrating a 16x8 macroblock for explaining usages of only pixels enclosing divided blocks in the case of deriving an average pixel value and a predicted average pixel value of the divided blocks.
  • an average value of the B16x8_0 block and the B16x8__l block can be represented by the following equation 8 : [Equation 8]
  • an average value of the B16x8_0 block can be represented by the following equation 9
  • an average value of the B16x8_l block can be represented by the following equation 10: [Equation 9]
  • an average pixel value of the B16x8_0 block of FIG. 6A can be represented by the following equation 11
  • the average pixel value of the B16x8_0 of FIG. 6B can be represented by the following equation 12: [Equation 11]
  • an average pixel value of the B16x8_l block of FIG. 6A can be represented by the following equation 13
  • the average pixel value of the B16x8_l of FIG. 6B can be represented by the following equation 14: [Equation 13]
  • FIG. 7A is a conceptual diagram illustrating a 8x16 macroblock for explaining usages of all the pixels enclosing divided blocks in the case of deriving an average pixel value and a predicted average pixel value of the divided blocks.
  • FIG. 7B is a conceptual diagram illustrating a 8x16 macroblock for explaining usages of only pixels enclosing divided blocks in the case of deriving an average pixel value and a predicted average pixel value of the divided blocks .
  • the method for deriving an average pixel value of the divided blocks is the same as
  • FIG. 8A is a conceptual diagram illustrating a 8x8 macroblock for explaining usages of all the pixels enclosing divided blocks in the case of deriving an average pixel value and a predicted average pixel value of the divided blocks.
  • FIG. 8B is a conceptual diagram illustrating a 8x8 macroblock for explaining usages of only pixels enclosing divided blocks in the case of deriving an average pixel value and a predicted average pixel value of the divided blocks.
  • the method for deriving an average pixel value of the divided blocks is the same as that of
  • FIGS. 6A-6B are identical to FIGS. 6A-6B.
  • the 8x8 block can be divided into a plurality of sub-blocks .
  • An average pixel value of a corresponding block of a current block of a current frame to be encoded is predicted
  • average pixel value is set to m r .
  • Equation 15 ( ⁇ x,Ay) is indicative of a disparity-
  • a reference block having a minimum block residual value is selected as an illumination-compensated optimum block.
  • the disparity vector is denoted by (Ax,Ay) .
  • an average pixel value of the reference block is not predicted by pixel values of neighboring blocks, and is directly calculated by an average pixel value of all pixels contained in an actual block.
  • the number of left- and upper-part pixels may be increased.
  • pixels of two or more neighboring layers of a current layer may be used instead of pixels of only one layer next to a current layer.
  • the decoding unit determines whether to perform an illumination compensation of a corresponding block using the ID information. If the illumination compensation is performed, the decoding unit calculates a decoded value of the difference value (e) , and obtains a predicted value according to an above-mentioned prediction method. The decoded value of the difference value (e) is added to the
  • the value of m r —m c is deducted from the reference block, which is prediction block so called predictor for the current block, and the deducted result is added to the decoded value of the residual block, such that the value of the current block can be finally obtained.
  • the current block can be reconstructed as follow:
  • B prediction block + residual block + ⁇ m c -m r + e) , where B is the value of the current block, reference block is the predictor for the current block, m.-m,- is a predicted difference of average pixel values, that is the predicted offset value of illumination compensation for the current block, and e is the difference value.
  • the decoding unit obtains the difference between a offset value of illumination compensation of the current block and a predicted difference, and can reconstruct the offset value of illumination compensation of the current block using the obtained residual block value and the predicted difference.
  • FIG. 9 is a diagram illustrating a process for obtaining an offset value of a current block.
  • the illumination compensation may be performed during the motion estimation. When it compares the current block with the reference block, a difference in illumination between two blocks is considered. New motion estimation and new motion compensation are used to compensate for the illumination difference.
  • a new SAD Sud of Absolute Differences
  • M c is indicative of an average pixel value of the current block
  • M r is indicative of an average pixel value of the
  • I c (x,y) is indicative of a pixel value at a
  • I r (x+ Ax,y+ Ay) is indicative of a pixel value at a motion
  • Equation 16 a difference value between an average pixel value of the current block and an average pixel value of the reference block can be obtained.
  • the difference value in average pixel value between the current block and the reference block is referred to as an offset value (IC_offset) .
  • the illumination compensation can be performed by the following equation 18 using the offset value and the motion vector: [Equation 18]
  • R(x,y) I c (x,y)-I,.(x+Ax,y+Ay)-(M c -M r )
  • the illumination compensation of the decoding unit can be performed by the following equation 19:
  • I c (x,y) is indicative of a pixel value of the
  • the offset value is transmitted to the decoding unit, and the offset value can be predicted by data of the neighboring blocks.
  • a difference value (R 10 offset ) between the current-block offset value (IC_offset) and the neighboring-block offset value (IC_offset_pred) can be transmitted to the decoding unit 50, as denoted by the following equation 20: [Equation 20 ]
  • FIG. 10 is a flow chart illustrating a process for performing for an illumination compensation of a current block.
  • an illumination compensation flag of a current block is set to "0"
  • the illumination compensation of the current block is not performed. Otherwise, if the illumination compensation flag of the current block is set to "1", a process for reconstructing the offset value of the current block is performed.
  • information of the neighboring block can be employed. It is determined whether a reference index of the current block is equal to a reference index of the neighboring block at step S210.
  • a predictor for performing the illumination compensation of the current block is obtained on the basis of the determined result at step S211.
  • An offset value of the current block is reconstructed by using the obtained predictor at step S212.
  • the step S210 for determining whether the reference index of the current block is equal to that of the neighboring block and the step S211 for obtaining the predictor on the basis of the determined result will hereinafter be described with reference to FIG. 11.
  • FIG. 11 is a flow chart illustrating a method for obtaining a predictor by determining whether a reference index of a current block is equal to a reference index of a neighboring block.
  • the decoding unit extracts a variety of information from a video signal, for example, flag information and offset values of neighboring blocks of the current block, and reference indexes of reference blocks of the current and neighboring blocks, such that the decoding unit can obtain the predictor of the current block using the extracted information.
  • the decoding unit obtains a residual value between the offset value of the current block and the predictor, and can reconstruct the offset value of the current block using the obtained residual value and the predictor.
  • information of the neighboring block can be employed.
  • the offset value of the current block can be predicted by the offset value of the neighboring block.
  • the reference index of the current block is equal to that of the neighboring block, such that it can be determined which one of values or which one of neighboring blocks will be used by referring to the determined result.
  • flag information of the neighboring block is set to "true", such that it can be determined whether the neighboring block will be used by referring to the determined result.
  • step S220 If it is determined that three neighboring blocks, each of which has the same reference index as that of the current block, exist at step S220, a median value of the offset values of the three neighboring blocks is assigned to the predictor of the current block at step S223. If it is determined that there is no neighboring block having the same reference index as that of the current block according to the determined result at step S220, the predictor of the current block is set to "0" at step S224. If required, the step S220 for determining whether the reference index of the current block is equal to that of the neighboring block may further include another step for determining whether a flag of the neighboring block is set to "1".
  • a plurality of neighboring blocks may be checked in the order of a left neighboring block -> an upper neighboring block - ⁇ a right-upper neighboring block -> a left-upper neighboring block. If required, the neighboring blocks may also be checked in the order of the upper neighboring block
  • the predictor of the current block may be set to "0".
  • FIG. 12 is a flow chart illustrating a method for performing for an illumination compensation on the basis of a prediction type of a current block.
  • the neighboring block acting as a reference block may be changed according to a prediction type of the current block. For example, if the current block has the same shape as that of the neighboring block, the current block is predicted by a median value of the neighboring blocks. Otherwise, if the shape of the current block is different from that of the neighboring block, another method will be employed. For example, if a block located at the left side of the current block is divided into several sub-blocks, the uppermost sub-block from among the sub-blocks is used for the prediction. Also, if a block located at an upper part of the current block is divided into several sub-blocks, the leftmost sub-block is used for the prediction.
  • a prediction value may be changed according to the prediction type of the current block. Therefore, the example of FIG. 12 determines a neighboring block to be referred by the prediction type of the current block at step S231. It is determined whether the reference index of the determined neighboring block is equal to a reference index of the current block at step S232.
  • the step S232 for determining whether the reference index of the neighboring block is equal to that of the current block may further include another step for determining whether a flag of the neighboring block is set to "1".
  • the predictor for performing an illumination compensation of the current block can be obtained on the basis of the determined result at step S233.
  • the offset value of the current block is reconstructed by the obtained predictor, such that the illumination compensation can be performed at step S234.
  • the process for performing the step S233 by referring to the result of step S232 will hereinafter be described in detail, and a detailed description thereof will be similar to that of FIG. 11.
  • the prediction type of the current block indicates that the prediction is performed by using a neighboring block located at the left side of the current block
  • the prediction type of the current block indicates that the prediction is performed by referring to the left- and upper- neighboring blocks of the current block, or if the prediction is performed by referring to three neighboring blocks (i.e., the left neighboring block, the upper neighboring block, and the right-upper neighboring block) , the individual cases will be applied similarly as a method of FIG. 11.
  • FIG. 13 is a flow chart illustrating a method for performing for an illumination compensation using flag information indicating whether the illumination compensation of a block is performed.
  • flag information (IC_flag) indicating whether an illumination compensation of the current block is performed may also be used to reconstruct the offset value of the current block.
  • the predictor may also be obtained using both the method for checking the reference index of FIG. 11 and the method for predicting flag information. Firstly, it is determined whether a neighboring block having the same reference index as that of the current block exists at step S241. A predictor for performing an illumination compensation of the current block is obtained by the determined result at step S242. In this case, a process for determining whether the flag of the neighboring block is "1" may also be included in the step S242. The flag information of the current block is predicted on the basis of the determined result at step S243.
  • step S242 An offset value of the current block is reconstructed by using the obtained predictor and the predicted flag information, such that the illumination compensation can be performed at step S244.
  • the step S242 may be applied similarly as a method of FIG. 11, and the step S243 will hereinafter be described with reference to FIG. 14.
  • FIG. 14 is a flow chart illustrating a method for predicting flag information of a current block by determining whether a reference index of the current block is equal to a reference index of a neighboring block. Referring to FIG. 14, it is determined whether the neighboring block having the same reference index as that of the current block exists at step S250. If it is determined that only one neighboring block having the same reference index as that of the current block exists, flag information of the current block is predicted by flag information of the neighboring block having the same reference index at step S251. If it is determined that two neighboring blocks, each of which has the same reference index as that of the current block, exist at step S250, flag information of the current block is predicted by any one of flag information of the two neighboring blocks having the same reference index at step S252.
  • the flag information of the current block is predicted by a median value of the flag information of the three neighboring blocks at step S253. Also, if there is no neighboring block having the same reference index as that of the current block according to the determined result of step S250, the flag information of the current block is not predicted at step S254.
  • FIG. 15 is a flow chart illustrating a method for performing an illumination compensation when a current block is predictively coded by two or more reference blocks.
  • the decoding unit cannot directly recognize an offset value corresponding to each reference block, because it uses an average pixel value of the two reference blocks when obtaining the offset value of the current block. Therefore, in one example, an offset value corresponding to each reference block is obtained, resulting in the implementation of correct prediction.
  • the offset value of the current block is reconstructed by using the predictor of the current block and the residual value at step S261. If the current block is predictively encoded by using two reference blocks, an offset value corresponding to each reference is obtained by the offset value at step S262, as denoted by the following equation 21: [Equation 21]
  • Equation 21 m c is an average pixel value of the
  • ⁇ r , and m r2 are indicative of an average
  • W 2 are indicative of a weighted coefficients for a bi-
  • the system independently obtains an accurate offset value corresponding to each reference block, such that it can more correctly perform the predictive coding process.
  • the system adds the reconstructed residual value and the predictor value, such that it obtains an offset value.
  • the predictor of a reference picture of ListO and the predictor of a reference picture of Listl are obtained repectively and combined, such that the system can obtain a predictor used for reconstructing the offset value of the current block.
  • the system can also be applied to skip-macroblock.
  • the prediction is performed to obtain an information for the illumination- compensation.
  • a value predicted by the neighboring block block is used as flag information indicating whether the illumination compensation is performed.
  • An offset value predicted by the neighboring block may be used as the offset value of the current block. For example, if flag information is set to "true", the offset value is added to a reference block.
  • the prediction is performed by using flags and offset values of the left- and upper- neighboring blocks, such that flag and offset values of the macroblock can be obtained.
  • a flag and an offset value of the current block may be set to the flag and the offset value of the block, respectively. If two blocks have the flag of "1”, the flag of the current block is set to "1", and the offset value of the current block is set to an average offset value of the two neighboring blocks.
  • the system can also be applied to a direct mode, for example, temporal direct mode, B-skip mode, etc.
  • the prediction is performed to obtain information of the illumination- compensation.
  • Each predictor can be obtained by using the variable method for predicting the flag and the offset. This predictor may be set to an actual flag and an actual offset value of the current block. If each block has a pair of flags and offset information, a prediction value for each block can be obtained. In this case, if there are two reference blocks and the reference indexes of the two reference blocks are checked, it is determined whether the reference index of the current block is equal to that of the neighboring block.
  • each reference block includes a unique offset value
  • first predicted flag information a first predicted offset value, second predicted flag information, and a second predicted offset value
  • a value predicted by the neighboring block may be used as the flag information.
  • the offset values of the two reference blocks may be used as the first predicted offset value and the second predicted offset value, respectively.
  • the offset value of the current block may be set to an average offset value of individual reference blocks.
  • the system may encode/decode the flag information indicating whether the direct mode or the skip-macroblock mode is applied to the current block.
  • an offset value is added or not according to the flag value.
  • a residual value between the offset value and the predicted offset value may also be encoded/decoded.
  • desired data can be more correctly reconstructed, and an optimum mode may be selected in consideration of a RD (Rate-Distortion) - relationship. If a reference picture cannot be used for the prediction process, i.e., if a reference picture number is less than "1", the flag information or predicted flag information may be set to "false", and the offset value or the predicted offset value may also be set to "0".
  • the system can also be applied to the entropy-coding process.
  • three context models may be used according to flag values of the neighboring blocks (e.g., blocks located at the left- and upper- parts of the current block) . If it is determined that the flag value is set to "true”, the value of "1" occurs. If it is determined that the flag value is set to "false", the value of "0" occurs. If the two values "1" and "0" of the two cases are added, three cases can be obtained.
  • the flag information is encoded/decoded by using the three context models.
  • a transform-coefficient level coding method can be used for the predictive residual value of the offset values.
  • data binarization is performed by UEGO
  • a single context model can be applied to a first bin value
  • another context mode is applied to the remaining bin values of a unary prefix part
  • a sign bit is encoded/decoded by a bypass mode.
  • two contexts may be considered according to a predicted flag values, such that the encoding/decoding process can be performed.
  • FIG. 16 is a flow chart illustrating a method for performing illumination compensation using not only flag information indicating whether illumination compensation of a current block is performed, but also an offset value of the current block.
  • the decoding unit extracts a variety of information from a video signal, for example, flag information and offset values of the current and neighboring blocks of the current block, and index information of reference blocks of the current and neighboring blocks, such that the decoding unit can obtain the predictor of the current block using the above- mentioned extracted information.
  • the decoding unit 50 obtains a residual value between the offset value of the current block and the predictor, and can reconstruct the offset value of the current block using the obtained residual value and the predictor.
  • flag information indicating whether the illumination compensation of the current block is performed may be used.
  • the decoding unit obtains flag information indicating whether the illumination compensation of the current block is performed at step S271. If the illumination compensation is performed according to the above-mentioned flag information (IC_flag) , the offset value of the current block indicating a difference in average pixel value between the current block and the reference block can be reconstructed at step S272. In this way, the above-mentioned illumination compensation technology encodes a difference value in average pixel value between blocks of different pictures. If a corresponding block is contained in the P slice when the flag indicating whether the illumination compensation is applied to each block, single flag information and a single offset value are encoded/decoded. However, if the corresponding block is contained in the B slice, a variety of methods can be made available, and a detailed description thereof will hereinafter be described with
  • FIGS. 17A-17B are diagrams illustrating a method for
  • C is indicative of a current block
  • N is indicative of a neighboring block of the current block (C)
  • R is indicative of a reference block of the current block (C)
  • S is indicative of a reference block of the neighboring block (N) of the current block (C)
  • m c is indicative of an average pixel value of the current block (C)
  • m r is indicative of an average pixel value of the reference block of the current block (C)
  • the encoding unit can transmit the residual value (R IC off set ) between the offset value (IC_offset) of the current block and the offset value (IC__offset_pred) of the neighboring block to a decoding unit, such that it can reconstruct the offset value "IC_offset” of the current block (C) .
  • the "Ric o f f set” information can also be represented by the above-mentioned Equation 20.
  • a variety of methods can be made available. For example, information of only one neighboring block may be employed, or information of two or more neighboring blocks may also be employed. In the case of employing the information of two or more neighboring blocks, an average value or a median value may be employed. In this way, if the current block is predictively encoded by a single reference block, the illumination compensation can be performed using a single offset value and single flag information. However, if the corresponding block is contained in the B slice, i.e., if the current block is predictively encoded by two or more reference blocks, a variety of methods can be made available. For example, as shown in FlG.
  • a predictor of the current block can be obtained by combining information of two reference blocks via the motion compensation.
  • single flag information indicates whether the illumination compensation of the current block is performed. If the flag information is determined to be "true”, a single offset value is obtained from the current block and the predictor, such that the encoding/decoding processes can be performed.
  • Flag information is assigned to each of the two reference blocks, and a single offset value obtained by using the above-mentioned flag information may be encoded or decoded.
  • two flag information may be used on the basis of the reference block, and a single offset value may be used on the basis of the current block.
  • single flag information may indicate whether the illumination compensation will be applied to a corresponding block on the basis of the current block.
  • Individual offset values can be encoded/decoded • for two reference blocks. If the illumination compensation is not applied to any one of the reference blocks during the encoding process, a corresponding offset value is set to "0".
  • single flag information may be used on the basis of the current block, and two offset values may be used on the basis of the reference block.
  • the flag information and the offset value can be encoded/decoded for individual reference blocks.
  • two flags and two offset values can be used on the bass of the reference block.
  • the offset value is not encoded without any change, and is predicted by an offset value of the neighboring block, such that its residual value is encoded.
  • FIG. 18 is a flow chart illustrating a method for performing an illumination compensation when a current block is predictively encoded by two or more reference blocks.
  • flag information and offset values of the neighboring blocks of the current block are extracted from the video signal, and index information of corresponding reference blocks of the current and neighboring blocks are extracted, such that the predictor of the current block can be obtained by using the extracted information.
  • the decoding unit obtains a residual value between the offset value of the current block and the predictor, and can reconstruct the offset value of the current block using the obtained residual value and the predictor.
  • flag information In the case of reconstructing the offset value of the current block, flag information
  • (IC_flag) indicating whether the illumination compensation of the current block is performed may be used as necessary.
  • the decoding unit obtains flag information indicating whether the illumination compensation of the current block is performed at step S291. If the illumination compensation is performed according to the above-mentioned flag information (IC_flag), the offset value of the current block indicating a difference in average pixel value between the current block and the reference block can be reconstructed at step S292.
  • Equation 22 m c is an average pixel value of the
  • m rl and m r2 are indicative of average pixel
  • the system independently obtains an accurate offset value corresponding to each reference block, such that it can more correctly perform the predictive coding process.
  • the system adds the reconstructed residual value and the predictor value, such that it obtains the offset value.
  • FIG. 19 is a flow chart illustrating a method for performing an illumination compensation using flag information indicating whether the illumination compensation of a current block is performed.
  • the illumination compensation technology is adapted to compensate for an illumination difference or a difference in color. If the scope of the illumination compensation technology is extended, the extended illumination compensation technology may also be applied between obtained sequences captured by the same camera. The illumination compensation technology can prevent the difference in illumination or color from greatly affecting the motion estimation. However, indeed, the encoding process employs flag information indicating whether the illumination compensation is performed.
  • the application scope of the illumination compensation may be extended to a sequence, a view, a GOP (Group Of Pictures) , a picture, a slice, a macroblock, and a sub-block, etc.
  • the illumination compensation technology is applied to a small-sized area, a local area may also be controlled, however, it should be noted that a large number of bits used for the flag information are consumed.
  • the illumination compensation technology may not be required. Therefore, a flag bit indicating whether the illumination compensation is assigned to individual areas, such that the system can effectively use the illumination compensation technology.
  • the system obtains flag information capable of allowing a specific level of the video signal to be illumination-compensated at step S201.
  • flag information may be assigned to individual areas. "seq_IC_flag” information is assigned to a sequence level, “view__IC_flag” information is assigned to a view level, “GOP_IC_flag” information is assigned to a GOP level, “pic_IC_flag” information is assigned to a picture level, “slice_IC_flag” information is assigned to a slice level, “mb_IC_flag” information is assigned to a macroblock level, and ⁇ 'blk_IC_flag” information is assigned to a block level.
  • FIGS. 20A-20C are conceptual diagrams illustrating
  • flag information indicating whether illumination compensation of a current block is performed. Referring to FIGS. 20A ⁇ 20c, the flag information
  • indicating whether the illumination compensation is performed can hierarchically be classified. For example,
  • N View_IC_flag” information 312 is assigned to a view level
  • GOP_IC_flag” information 313 is assigned to a GOP level
  • picture_IC_flag” information 314 is assigned to a picture level
  • slice_IC_flag” information 315 is assigned to a slice level
  • mb_IC_flag” information 316 is assigned to a macroblock level
  • blk_IC_flag” information 317 is assigned to a block level.
  • each flag is composed of 1 bit.
  • the number of the above-mentioned flags may be set to at least one.
  • the above-mentioned sequence/view/picture/slice-level flags may be located at a corresponding parameter set or header, or may also be located another parameter set.
  • the "seq_IC_flag" information 311 may be located at a sequence parameter set
  • the "view_IC_flag” information 312 may be located at the view parameter set
  • the M pic__IC_flag” information 314 may be located at the picture parameter set
  • the "slice_IC_flag” information 315 may be located at the slice header.
  • specific information indicating whether the illumination compensation of an upper level is performed may control whether the illumination compensation of a lower level is performed. In other words, if each flag bit value is set to "1", the illumination compensation technology may be applied to a lower level.
  • the "pic_IC_flag” information is set to “1”
  • the ⁇ N slice__IC_flag” information of each slice contained in a corresponding picture may be set to “1” or w 0”
  • the "mb_IC_flag” information of each macroblock may be set to “1” or “0”
  • the "blk_IC_flag” information of each block may be set to “1” or “0”.
  • the "seq_IC_flag” information is set to N ⁇ 1" on the condition that a view parameter set exists, the "view_IC_flag” value of each view may be set to "1” or "0".
  • a flag bit value of GOP, picture, slice, macroblock, or block of a corresponding view may be set to "1" or xx 0", as shown in FIG. 2OA.
  • the above-mentioned flag bit value of GOP, picture, slice, macroblock, or block of the corresponding view may not be set to "1” or u 0" as necessary.
  • the above-mentioned flag bit value of GOP, picture, slice, macroblock, or block of the corresponding view may not be set to "1" or "0", this indicates that the GOP flag, the picture flag, the slice flag, the macroblock flag, or the block flag is not controlled by the view flag information, as shown in FIG 2OB.
  • the flag bit values of a lower scope are automatically set to "0". For example, if the xx seq_IC_flag” information is set to "0", this indicates that the illumination compensation technology is not applied to a corresponding sequence. Therefore, the xx view_IC_flag” information is set to "0", the xx GOP_IC_flag” information is set to “0”, the “pic_IC_flag” information is set to “0”, the “slice_IC_flag” information is set to xx 0", the xx mb_IC_flag” information is set to "0”, and the xx blk_IC_flag” information is set to "0".
  • mb_IC_flag information or only one xx blk__IC_flag” information may be employed according to a specific implementation methods of the illumination compensation technology.
  • the "view_IC_flag" information may be employed when the view parameter set is newly applied to the multiview video coding.
  • the offset value of the current block may be additionally encoded/decoded according to a flag bit value of the macroblock or sub- block acting as the lowest-level unit.
  • the flag indicating the IC technique application may also be applied to both the slice level and macroblock level. For example, if the "slice_IC_flag" information is set to “0”, this indicates that the IC technique is not applied to a corresponding slice. If the “slice__IC_flag” information is set to "1”, this indicates that the IC technique is applied to a corresponding slice. In this case, if the "mb_IC_flag” information is set to ⁇ X l", “IC__offset” information of a corresponding macroblock is reconstructed. If the "mb_IC_flag” information is set to "0”, this indicates that the IC technique is not applied to a corresponding macroblock.
  • the system can obtain an offset value of a current block indicating a difference in average pixel value between the current block and the reference block.
  • the flag information of the macroblock level or the flag information of the block level may not be employed as necessary.
  • the illumination compensation technique can indicate whether the illumination compensation of each block is performed using the flag information.
  • the illumination compensation technique may also indicate whether the illumination compensation of each block is performed using a specific value such as a motion vector. The above-mentioned example can also be applied to a variety of applications of the illumination compensation technique.
  • the above-mentioned example can indicate whether the illumination compensation of a lower scope is performed using the flag information.
  • the macroblock or block level acting as the lowest scope can effectively indicate whether the illumination compensation is performed using the offset value without using the flag bit.
  • the predictive coding process can be performed. For example, if the predictive coding process is applied to the current block, the offset value of the neighboring block is assigned to an offset value of the current block. If the predictive coding scheme is determined to be the bi-predictive coding scheme, offset values of individual reference blocks are obtained by the calculation of the reference blocks detected from List 0 and List 1.
  • the offset value of each reference is not directly encoded by the offset values of the neighboring blocks, and a residual value is encoded/decoded.
  • the method for predicting the offset value may be determined to be the above-mentioned oftset prediction method or a method for obtaining a median value used for predicting the motion vector.
  • a direct mode of a bi-directional prediction supplementary information is not encoded/decoded using the same method as in the motion vector, and the offset values can be obtained by predetermined information.
  • a decoding unit e.g., H.264-based decoding unit
  • a view sequence compatible with a conventional decoding unit should be decoded by the conventional decoding unit, such that the "view_IC_flag" information is set to "false” or "0".
  • the base view is indicative of a reference view from among several views (i.e., the multiview) .
  • a sequence corresponding to the base view in the MVC scheme is encoded by general video encoding schemes (e.g., MPEG-2, MPEG-4, H.263, and H.264, etc.), such that it is generated in the form of an independent bitstream.
  • general video encoding schemes e.g., MPEG-2, MPEG-4, H.263, and H.264, etc.
  • the above-mentioned base-view sequence can be compatible with the H.264/AVC scheme, or cannot be compatible with the same. However, the view sequence compatible with the H.264/AVC scheme is always set to the base view.
  • FIG. 21 is a flow chart illustrating a method for obtaining a motion vector considering an offset value of a current block.
  • the system can obtain an offset value of the current block at step S321.
  • the system searches for a reference block optimally matched with the current block using the offset value at step S322.
  • the system obtains the motion vector from the reference block, and encodes the motion vector at step S323.
  • For the illumination compensation a variety of factors -are considered during the motion estimation. For example, in the case of a method for comparing a first block with a second block by offsetting average pixel values of the first and second blocks, average pixel values of the two blocks are deducted from pixel values of each block during the motion estimation, such that the similarity between the two blocks can be calculated.
  • the offset value between the two blocks is independently encoded, such that the costs for the independent encoding are reflected in the motion estimation process.
  • the conventional costs can be calculated by the following equation 23: [Equation 23 ]
  • I c is indicative of a pixel value of
  • M c is indicative of an average pixel
  • a is indicative of a weighted coefficient. If the value of a is set to "1", the absolute value of the offset value is reflected. For another method for reflecting the illumination compensation cost, there is a method for reflecting the illumination compensation cost by predicting the number of bits required for encoding the offset value.
  • the following equation 27 represents a method for predicting the offset coding bit.
  • the coding bit can be predicted in proportion to the magnitude of an offset residual value.

Abstract

Decoding a multiview video signal comprises: receiving a bitstream comprising encodings of multiple views of the multiview video signal. Each view comprises multiple pictures segmented into multiple segments. The decoding also comprises extracting flag information associated with a portion of the multiview video signal from the bitstream indicating whether illumination compensation of segments within said portion of the multiview video signal is enabled. For a portion in which illumination compensation is enabled according to the extracted flag information, a value associated with a segment within the portion is extracted from the bitstream and it is determined from said extracted value whether illumination compensation of the segment is to be performed.

Description

PROCESSINGMULTIVIEWVIDEO
Technical Field
The invention relates to processing multiview video.
Background Art
Multiview Video Coding (MVC) relates to compression of video sequences (e.g., a sequence of images or
"pictures") that are typically acquired by respective cameras. The video sequences or "views" can be encoded according to a standard such as MPEG. A picture in a video sequence can represent a full video frame or a field of a video frame. A slice is an independently coded portion of a picture that includes some or all of the macroblocks in the picture, and a macroblock includes blocks of picture elements (or "pixels").
The video sequences can be encoded as a multiview video sequence according to the H.264/AVC codec technology, and many developers are conducting research into amendment of standardis to accomodate multiview video sequences.
Three profiles for supporting specific functions are prescribed in the current H.264 standard. The term "profile" indicates the standardization of technical components for use in the video encoding/decoding algorithms. In other words, the profile is the set of technical components prescribed for decoding a bitstream of a compressed sequence, and may be considered to be a substandard. The above-mentioned three profiles are a baseline profile, a main profile, and an extended profile. A variety of functions for the encoder and the decoder have been defined in the H.264 standard, such that the encoder and the decoder can be compatible with the baseline profile, the main profile, and the extended profile respectively. The bitstream for the H.264/AVC standard is structured according to a Video Coding Layer (VCL) for processing the moving-image coding (i.e., the sequence coding) , and a Network Abstraction Layer (NAL) associated with a subsystem capable of transmitting/storing encoded information. The output data of the encoding process is VCL data, and is mapped into NAL units before it is transmitted or stored. Each NAL unit includes a Raw Byte Sequence Payload (RBSP) corresponding to either compressed video data or header information. The NAL unit includes a NAL header and a RBSP. The NAL header includes flag information (e.g., nal__ref_idc) and identification (ID) information (e.g., nal_unit__type) . The flag information "nal__ref_idc" indicates the presence or absence of a slice used as a reference picture of the NAL unit. The ID information "nal_unit_type" indicates the type of the NAL unit. The RBSP stores compressed original data. An RBSP trailing bit can be added to the last part of the RBSP, such that the length of the RBSP can be represented by a multiple of 8 bits.
There are a variety of the NAL units, for example, an Instantaneous Decoding Refresh (IDR) picture, a Sequence Parameter Set (SPS), a Picture Parameter Set (PPS), and Supplemental Enhancement Information (SEI), etc. The standard has generally defined a target product using various profiles and levels, such that the target product can be implemented with appropriate costs. The decoder satisfies a predetermined constraint at a corresponding profile and level . The profile and the level are able to indicate a function or parameter of the decoder, such that they indicate which compressed images can be handled by the decoder. Specific information indicating which one of multiple profiles corresponds to the bitstream can be identified by profile ID information. The profile ID information "profile_idc" provides a flag for identifying a profile associated with the bitstream. The H.264/AVC standard includes three profile identifiers (IDs). If the profile ID information "profile idc" is set to "66", the bitstream is based on the baseline profile. If the profile ID information "profile__idc" is set to "77", the bitstream is based on the main profile. If the profile ID information "profile__idc" is set to "88", the bitstream is based on the extended profile. The above-mentioned "profile_idc" information may be contained in the SPS (Sequence Parameter Set), for example. Disclosure of Invention
In one aspect, in general, a method for decoding a multiview video signal comprises: receiving a bitstream comprising encodings of multiple views of the multiview video signal, each view comprising multiple pictures segmented into multiple segments (e.g., an image block segment such as a single block or a macroblock, or a segment such as a slice of an image) ; extracting flag information associated with a portion of the multiview video signal from the bitstream indicating whether illumination compensation of segments within said portion of the multiview video signal is enabled; and for a portion in which illumination compensation is enabled according to the extracted flag information, extracting from the bitstream a value associated with a segment within the portion and determining from said extracted value whether illumination compensation of the segment is to be performed, Aspects can include one or more of the following features .
The segments comprise image blocks.
The method further comprises, for a first block associated with a value that indicates that illumination compensation is to be performed, obtaining a predictor for performing illumination compensation of the first block using an offset value for illumination compensation of at least one neighboring block adjacent to the first block. An offset value for illumination compensation of a neighboring block is obtained by forming a sum that includes a predictor for illumination compensation of the neighboring block and a residual value.
Obtaining a predictor for illumination compensation of the first block using an offset value for illumination compensation of at least one neighboring block adjacent to the first block includes selecting the at least one neighboring block according to a predetermined order among the neighboring blocks. Selecting the at least one neighboring block according to the predetermined order comprises determining whether one or more conditions are satisfied for a neighboring block in an order in which one or more vertical or horizontal neighbors are followed by one or more diagonal neighbors.
The flag information enables illumination compensation for one or more of a sequence, a view, a group of pictures, a picture, and a slice that contains the block.
The flag information enables illumination compensation for the slice that contains the block.
The extracted value comprises flag information for a macroblock that contains the block or flag information for the block.
The extracted value comprises flag information for the macroblock that contains the block.
In another aspect, in general, a method for decoding a multiview video signal comprises: receiving a bitstream comprising encodings of multiple views of the multiview video signal, each view comprising multiple pictures segmented into multiple segments; and obtaining a predictor for illumination compensation of a first segment using an offset value for illumination compensation of at least one neighboring segment adjacent to the first segment, including selecting the at least one neighboring segment according to a predetermined order among the neighboring segments . Aspects can include one or more of the following features .
The first segment and the at least one neighboring segment comprise image blocks. An offset value for illumination compensation of a neighboring block is obtained by forming a sum that includes a predictor for illumination compensation of the neighboring block and a residual value.
Selecting the at least one neighboring block according to the predetermined order comprises determining whether one or more conditions are satisfied for a neighboring block in an order in which one or more vertical or horizontal neighbors are followed by one or more diagonal neighbors. Selecting the at least one neighboring block according to the predetermined order comprises determining whether one or more conditions are satisfied for a neighboring block in the order of: a left neighboring block, followed by an upper neighboring block, followed by a right-upper neighboring block, followed by a left-upper neighboring block.
Selecting the at least one neighboring block according to the predetermined order comprises determining whether one or more conditions are satisfied for a neighboring block in the order of: a upper neighboring block, followed by a left neighboring block, followed by a right-upper neighboring block, followed by a left-upper neighboring block. Determining whether one or more conditions are satisfied for a neighboring block comprises extracting a value associated with the neighboring block from the bitstream indicating whether illumination compensation of the neighboring block is to be performed. The extracted value comprises flag information for a macroblock that contains the block or flag information for the block.
Obtaining the predictor comprises determining whether to use an offset value for illumination compensation of a single neighboring block or multiple offset values for illumination compensation of respective neighboring blocks .
The method further comprises, when multiple offset values are to be used, obtaining the predictor for performing illumination compensation of the first block by combining the multiple offset values.
Combining the multiple offset values comprises taking an average or median of the offset values. In another aspect, in general, a method for decoding a multiview video signal comprises: receiving a bitstream comprising encodings of multiple views of the multiview video signal, each view comprising multiple pictures segmented into multiple segments; obtaining an offset value for illumination compensation of a first segment with respect to a reference picture, wherein the offset value is predicted using an offset value for illumination compensation of at least one neighboring segment determined based on characteristics associated with the neighboring segment; and decoding the bitstream using illumination compensation for the first segment including forming a sum that includes a predictor for pixels of the first segment obtained from the reference picture, a residual for pixels of the first segment, and a corresponding offset value for illumination compensation.
Aspects can include one or more of the following features .
The first segment and the at least one neighboring segment comprise image blocks.
An offset value for illumination compensation of a neighboring block is obtained by forming a sum that includes a predictor for illumination compensation of the neighboring block and a residual value. The method further comprises selecting at least one neighboring block based on whether one or more conditions are satisfied for a neighboring block in an order in which one or more vertical or horizontal neighbors are followed by one or more diagonal neighbors .
Selecting at least one neighboring block comprises determining whether one or more conditions are satisfied for a neighboring block in the order of: a left neighboring block, followed by an upper neighboring block, followed by a right-upper neighboring block, followed by a left-upper neighboring block.
Determining whether one or more conditions are satisfied for a neighboring block comprises extracting a value associated with the neighboring block from the bitstream indicating whether illumination compensation of the neighboring block is to be performed.
The extracted value comprises flag information for a macroblock that contains the block or flag information for the block. Selecting at least one neighboring block comprises determining whether to use an offset value for illumination compensation of a single neighboring block or multiple offset values for illumination compensation of respective neighboring blocks. The method further comprises, when multiple offset values are to be used, obtaining the predictor for performing illumination compensation of the first block by combining the multiple offset values . Combining the multiple offset values comprises taking an average or median of the offset values.
In another aspect, in general, a method for decoding a multiview video signal, comprises: receiving a bitstream comprising encodings of multiple views of the multiview video signal, each view comprising multiple pictures segmented into multiple segments; obtaining a predictor for illumination compensation of a first segment with respect to a reference picture; determining an offset value for illumination compensation of the first segment including forming a sum that includes the predictor and a residual value; and decoding the bitstream using illumination compensation for the first segment including forming a sum that includes a predictor for pixels of the first segment obtained from the reference picture, a residual for pixels of the first segment, and a corresponding offset value for illumination compensation.
Aspects can include one or more of the following features .
The segments comprise image blocks. Using illumination compensation for the first segment comprises obtaining an offset value for illumination compensation of a neighboring block by forming a sum that includes a predictor for illumination compensation of the neighboring block and a residual value.
The method further comprises selecting at least one neighboring block based on whether one or more conditions are satisfied for a neighboring block in an order in which one or more vertical or horizontal neighbors are followed by one or more diagonal neighbors.
Selecting at least one neighboring block comprises determining whether one or more conditions are satisfied for a neighboring block in the order of: a left neighboring block, followed by , an upper neighboring block, followed by a right-upper neighboring block, followed by a left-upper neighboring block.
Determining whether one or more conditions are satisfied for a neighboring block comprises extracting a value associated with the neighboring block from the bitstream indicating whether illumination compensation of the neighboring block is to be performed.
The extracted value comprises flag information for a macroblock that contains the block or flag information for the block. Selecting at least one neighboring block comprises determining whether to use an offset value for illumination compensation of a single neighboring block or multiple offset values for illumination compensation of respective neighboring blocks.
The method further comprises, when multiple offset values are to be used, obtaining the predictor for performing illumination compensation of the first block by combining the multiple offset values. Combining the multiple offset values comprises taking an average or median of the offset values.
In another aspect, in general, a method for decoding a multiview video signal comprises: receiving a bitstream comprising encodings of multiple views of the multiview video signal, each view comprising multiple pictures segmented into multiple segments; and obtaining a predictor for illumination compensation of a first segment with respect to a reference picture using an offset value for illumination compensation of at least one neighboring segment adjacent to the first segment according whether the reference picture associated with the first segment is the same as a reference picture associated with the neighboring segment . Aspects can include one or more of the following features .
The segments comprise image blocks.
Using illumination compensation for the first segment comprises obtaining an offset value for illumination compensation of a neighboring block by forming a sum that includes a predictor for illumination compensation of the neighboring block and a residual value.
The method further comprises selecting at least one neighboring block based on whether one or more conditions are satisfied for a neighboring block in an order in which one or more vertical or horizontal neighbors are followed by one or more diagonal neighbors.
Selecting at least one neighboring block comprises determining whether one or more conditions are satisfied for a neighboring block in the order of: a left neighboring block, followed by an upper neighboring block, followed by a right-upper neighboring block, followed by a left-upper neighboring block. Determining whether one or more conditions are satisfied for a neighboring block comprises extracting a value associated with the neighboring block from the bitstream indicating whether illumination compensation of the neighboring block is to be performed. The extracted value comprises flag information for a macroblock that contains the block or flag information for the block.
Selecting at least one neighboring block comprises determining whether to use an offset value for illumination compensation of a single neighboring block or multiple offset values for illumination compensation of respective neighboring blocks.
The method further comprises, when multiple offset values are to be used, obtaining the predictor for performing illumination compensation of the first block by combining the multiple offset values .
Combining the multiple offset values comprises taking an average or median of the offset values. In another aspect, in general, for each respective decoding method, a method for encoding a video signal comprises generating a bitstream capable of being decoded into the video signal by the respective decoding method.
For example, in another aspect, in general, a method for encoding a bitstream comprises: forming the bitstream comprising encodings of multiple views of the multiview video signal, each view comprising multiple pictures segmented into multiple segments; providing flag information associated with a portion of the multiview video signal in the bitstream indicating whether illumination compensation of segments within said portion of the multiview video signal is enabled; and for a portion in which illumination compensation is enabled according to the extracted flag information, providing in the bitstream a value associated with a segment within the portion and determining from said extracted value whether illumination compensation of the segment is to be performed.
In another aspect, in general, a method for encoding a bitstream comprises: forming the bitstream comprising encodings of multiple views of the multiview video signal, each view comprising multiple pictures segmented into multiple segments; and providing a predictor for illumination compensation of a first segment using an offset value for illumination compensation of at least one neighboring segment adjacent to the first segment, including selecting the at least one neighboring segment according to a predetermined order among the neighboring segments . In another aspect, in general, a method for encoding a bitstream comprises: forming the bitstream comprising encodings of multiple views of the multiview video signal, each view comprising multiple pictures segmented into multiple segments; providing an offset value for illumination compensation of a first segment with respect to a reference picture, wherein the offset value is able to be predicted using an offset value for illumination compensation of at least one neighboring segment determined based on characteristics associated with the neighboring segment; and providing information for illumination compensation for the first segment based on a sum that, includes a predictor for pixels of the first segment obtained from the reference picture, a residual for pixels of the first segment, and a corresponding offset value for illumination compensation.
In another aspect, in general, a method for encoding a bitstream comprises: forming the bitstream comprising encodings of multiple views of the multiview video signal, each view comprising multiple pictures segmented into multiple segments; providing a predictor for illumination compensation of a first segment with respect to a reference picture; providing an offset value for illumination compensation of the first segment based on a sum that includes the predictor and a residual value; and providing information for illumination compensation for the first segment based on a sum that includes a predictor for pixels of the first segment obtained from the reference picture, a residual for pixels of the first segment, and a corresponding offset value for illumination compensation.
In another aspect, in general, a method for encoding a bitstream comprises: forming the bitstream comprising encodings of multiple views of the multiview video signal, each view comprising multiple pictures segmented into multiple segments; and providing a predictor for illumination compensation of a first segment with respect to a reference picture using an offset value for illumination compensation of at least one neighboring segment adjacent to the first segment according whether the reference picture associated with the first segment is the same as a reference picture associated with the neighboring segment . In another aspect, in general, for each respective decoding method, a computer program, stored on a computer- readable medium, comprises instructions for causing a computer to perform the respective decoding method.
In another aspect, in general, for each respective decoding method, image data embodied on a machine-readable information carrier is capable of being decoded into a video signal by the respective decoding method. In another aspect, in general, for each respective decoding method, a decoder comprises means for performing the respective decoding method.
In another aspect, in general, for each respective decoding method, an encoder comprises means for generating a bitstream capable of being decoded into a video signal by the respective decoding method.
In another aspect, in general, a method for encoding a video sequence comprises: a) obtaining an average pixel value of at least one block from among neighboring blocks of a current block and reference blocks of another view; b) deriving a predicted average pixel value of the current block from the obtained average pixel value of the at least one block; and c) obtaining a difference value between a predicted average pixel value of the current block and an average pixel value of the current block.
In another aspect, in general, there is provided a method for decoding a video sequence comprising: 1) obtaining a difference value capable of reconstructing an average pixel value of a current block from a video signal; m) deriving a predicted average pixel value of the current block from reference blocks of another view; and n) reconstructing the average pixel value of the current block on the basis of the predicted average pixel value and the difference value.
In yet another aspect, in general, there is provided an apparatus for encoding a video sequence comprising: an average pixel value obtaining unit for obtaining average pixel values of neighboring blocks of a current block and reference blocks of another view; an average pixel value prediction unit for deriving a predicted average pixel value of the current block from the obtained average pixel value; and a differntial-value encoding unit for obtaining a difference value between the predicted average pixel value and the average pixel value of the current block.
In yet another aspect, in general, there is provided an apparatus for decoding a video sequence comprising: a difference-value decoding unit for obtaining a difference value from a received bitstream; an average pixel value prediction unit for deriving a predicted average pixel value of a current block from a reference block of another view; and an illumination compensation unit for reconstructing the average pixel value of the current block on the basis of the predicted average pixel value and the difference value.
In yet another aspect, in general, a method for decoding a video signal comprises: obtaining a predictor for performing illumination compensation of a current block using an offset value of at least one neighboring block adjacent to the current block; and reconstructing an offset value of the current block using the predictor, wherein the predictor is determined by determining whether a reference index of the current block is equal to a reference index of the neighboring block.
In yet another aspect, in general, there is provided a method for decoding a video signal comprising: reconstructing a current block offset value indicating a difference between an average pixel value of a current block and an average pixel value of at least one reference block; and obtaining respectively an offset values of the reference blocks of the current block using the offset value, if the current block is predictively encoded by two or more reference blocks .
In yet another aspect, in general, a method for decoding a video signal comprises: obtaining flag information indicating whether illumination compensation of a current block is performed; and if the illumination compensation is performed by the flag information, reconstructing an offset value indicating a difference between an average pixel value of the current block and an average pixel value of a reference block. In yet another aspect, in general, there is provided a method for decoding a video signal comprising: a) obtaining flag information for allowing a specific level of a video signal to be illumination-compensated; and b) decoding a specific level of the video signal illumination- compensated by the flag information, wherein the specific level of the video signal corresponds to any one of a sequence level, a view level, a GOP (Group of Pictures) level, a picture level, a slice level, a macroblock level, and a block level.
In yet another aspect, in general, there is provided a method for encoding a video signal comprising: obtaining a current-block' s offset value indicating a difference between an average pixel value of the current block and a reference block; and searching for a reference optimally matched with the current block using the offset value; and obtaining a motion vector from the matched reference block, and encoding the motion vector.
Aspects can have one or more of the following advantages .
The method or apparatus for encoding/decoding a video sequence predicts an average value of a current block to be encoded on the basis of peripheral blocks, and transmits a difference value between the current block and the peripheral blocks, thereby minimizing an amount of information to be transmitted for illumination compensation. The method effectively performs illumination compensation of a multiview video sequence requiring a large amount of data, thereby increasing an encoding rate. The method implements an effective encoding/decoding system using correlation between blocks or views.
View sequences of the multiview video data are captured by different cameras, such that there is a difference in illumination due to inner or outer factors of the cameras. In order to solve the problems, the method predicts an offset value of a current block using information of the neighboring block, transmits only a residual value between the current block and the neighboring block, such that it can minimize an amount of information to be transmitted for illumination compensation. In the case of predicting the offset value of the current block, the method determines whether a reference index of the current block is equal to that of the neighboring block, resulting in the implementation of correct prediction.
The method predicts flag information indicating whether the illumination compensation of the current block is performed, and transmits only a residual value between the flag information, thereby minimizing an amount of information to be transmitted. The method determines whether a reference index of the current block is equal to that of the neighboring block, resulting in the implementation of correct prediction. The method uses correlation between blocks or views, resulting in the implementation of the effective coding process.
View sequences of the multiview video data are captured by different cameras, such that there is a difference in illumination due to inner or outer factors of the cameras. In order to solve the above-mentioned problems, the method predicts an offset value of a current block using information of the neighboring block, transmits only a residual value between the current block and the neighboring block, such that it can minimize an amount of information to be transmitted for illumination compensation. The method predicts flag information indicating whether the illumination compensation of the current block is performed, and transmits only a residual value, thereby minimizing an amount of information to be transmitted. If the predictive coding process is performed using two or more reference blocks, the method employs the offset value and the flag information using at least one method, resulting in the implementation of the effective coding process. A flag bit indicating whether the illumination compensation is performed to each area of the video signal is assigned, such that the illumination compensation technique can be effectively used. The method calculates the costs by reflecting an illumination difference in the motion estimation process, resulting in the implementation of the correct predictive coding.
Other features and advantages will become apparent from the following description, and from the claims.
Brief Description of Drawings
FIG. 1 is an exemplary decoding apparatus. FIG. 2 is a flowchart illustrating a method for encoding a video sequence.
FIG. 3 is a block diagram illustrating a process for deriving a predicted average pixel value of a current block from reference blocks of other views.
FIG. 4 is a detailed block diagram illustrating a process for deriving a predicted average pixel value of a current block from reference blocks of other views. FIG. 5 is a diagram illustrating a 16x16 macroblock. FIGS. 6A-6B are diagrams illustrating 16x8 macroblocks .
FIGS. 7A-7B are diagrams illustrating 8x16 macroblocks . FIGS. 8A-8B are diagrams illustrating 8x8 macroblocks .
FIG. 9 is a diagram illustrating a process for obtaining an offset value of a current block. FIG. 10 is a flowchart illustrating a process for performing illumination compensation of a current block.
FIG. 11 is a flowchart illustrating a method for obtaining a predictor by determining whether a reference index of a current block is equal to a reference index of a neighboring block.
FIG. 12 is a flow chart illustrating a method for performing for an illumination compensation on the basis of a prediction type of a current block.
FIG. 13 is a flow chart illustrating a method for performing illumination compensation using flag information indicating whether the illumination compensation of a block is performed.
FIG. 14 is a flow chart illustrating a method for predicting flag information of a current block by determining whether a reference index of the current block is equal to a reference index of a neighboring block.
FIG. 15 is a flow chart illustrating a method for performing illumination compensation when a current block is predictively coded by two or more reference blocks. FIG. 16 is a flow chart illustrating a method for performing illumination compensation using not only a flag indicating whether illumination compensation of a current block is performed, but also an offset value of a current block.
FIGS. 17A-17B are diagrams illustrating a method for performing illumination compensation using a flag and an offset value in association with blocks of P and B slices.
FIG. 18 is a flow chart illustrating a method for performing illumination compensation when a current block is predictively encoded by two or more reference blocks.
FIG. 19 is a flow chart illustrating a method for performing illumination compensation using a flag indicating whether illumination compensation of a current block is performed.
FIGS. 20A-20C are diagrams illustrating the scope of flag information indicating whether illumination compensation of a current block is performed.
FIG. 21 is a flow chart illustrating a method for obtaining a motion vector considering an offset value of a current block.
Best Mode for Carrying Out the Invention
In order to effectively handle a multiview sequence, an input bitstream includes information that allows a decoding apparatus to determine whether the input bitstream relates to a multiview profile. In cases that it is determined that the input bitstream relates to the multiview profile, supplementary information associated with the multiview sequence is added according to a syntax to the bitstream and transmitted to the decoder. For example, the multiview profile ID can indicate a profile mode for handling multiview video data as according to an amendment of the H.264/AVC standard.
The MVC (Multiview Video Coding) technology is an amendment technology of the H.264/AVC standards. That is, a specific syntax is added as supplementary information for an MVC mode. Such amendment to support MVC technology can be more effective than an alternative in which an unconditional syntax is used. For example, if the profile identifier of the AVC technology is indicative of a multiview profile, the addition of multiview sequence information may increase a coding efficiency. The sequence parameter set (SPS) of the H.264/AVC bitstream is indicative of header information including information (e.g., a profile, and a level) associated with the entire-sequence encoding.
The entire compressed moving images (i.e., a sequence) can begin at a sequence header, such that a sequence parameter set (SPS) corresponding to the header information arrives at the decoder earlier than data referred to by the parameter set. As a result, the sequence parameter set RBSP acts as header information of a compressed data of moving images at entry Sl (FIG. 2) . If the bitstream is received, the profile ID information "profile_idc" identifies which one of profiles from among several profiles corresponds to the received bitstream. The profile ID information "profile_idc" can be set, for example, to "MULTI_VIEW_PROFILE) ", so that the syntax including the profile ID information can determine whether the received bitstream relates to a multiview profile. The following configuration information can be added when the received bitstream relates to the multiview profile.
FIG. 1 is a block diagram illustrating an exemplary decoding apparatus (or "decoder") of a multiview video system for decoding a video signalcontaining a multiview video sequence. The multiview video system includes a corresponding encoding apparatus (or "encoder") to provide the multiview video sequence as a bitstream that includes encoded image data embodied on a machine-readable information carrier (e.g., a machine-readable storage medium, or a machine-readable energy signal propagating between a transmitter and receiver. )
Referring to FIG. 1, the decoding apparatus includes a parsing unit 10, an entropy decoding unit 11, an Inverse
Quantization/Inverse Transform unit 12, an inter-prediction unit 13, an intra-prediction unit 14, a deblocking filter
15, and a decoded-picture buffer 16.
The inter-prediction unit 13 includes a motion compensation unit 17, an illumination compensation unit 18, and an illumination-compensation offset prediction unit 19. The parsing unit 10 performs a parsing of the received video sequence in NAL units to decode the received video sequence. Typically, one or more sequence parameter sets and picture parameter sets are transmitted to a decoder before a slice header and slice data are decoded. In this case, the NAL header or an extended area of the NAL header may include a variety of configuration information, for example, temporal level information, view level information, anchor picture ID information, and view ID information, etc. In this case, the term "time level information" is indicative of hierarchical-structure information for providing temporal scalability from a video signal, such that sequences of a variety of time zones can be provided to a user via the above-mentioned temporal level information.
The term "view level information" is indicative of hierarchical-structure information for providing view scalability from the video signal. The multiview video sequence can define the temporal level and view level, such that a variety of temporal sequences and view sequences can be provided to the user according to the defined temporal level and view level.
In this way, if the level information is defined as described above, the user may employ the temporal scalability and the view scalability. Therefore, the user can view a sequence corresponding to a desired time and view, or can view a sequence corresponding to another limitation. The above-mentioned level information may also be established in various ways according to reference conditions. For example, the level information may be changed according to a camera location, and may also be changed according to a camera arrangement type. In addition, the level information may also be arbitrarily established without a special reference.
The term "anchor picture" is indicative of an encoded picture in which all slices refer to only slices in a current view and not slices in other views. A random access between views can be based on anchor pictures for multiview-sequence decoding.
Anchor picture ID information can be used to perform the random access process to access data of a specific view without requiring a large amount of data to be decoded. The term "view ID information" is indicative of specific information for discriminating between a picture of a current view and a picture of another view. In order to discriminate one picture from other pictures when the video sequence signal is encoded, a Picture Order Count (POC) and frame number information (frame_num) can be used.
If a current sequence is determined to be a multiview video sequence, inter-view prediction can be performed. An identifier is used to discriminate a picture of the current view from a picture of another view. A view identifier can be defined to indicate a picture's view. The decoding apparatus can obtain information of a picture in a view different from a view of the current picture using the above-mentioned view identifier, such that it can decode the video signal using the information of the picture. The above-mentioned view identifier can be applied to the overall encoding/decoding process of the video signal. Also, the above-mentioned view identifier can also be applied to the multiview video coding process using the frame number information "frame_num" considering a view.
Typically, the multiview sequence has a large amount of data, and a hierarchical encoding function of each view
(also called a "view scalability") can be used for processing the large amount of data. In order to perform the view scalability function, a prediction structure considering views of the multiview sequence may be defined.
The above-mentioned prediction structure may be defined by structuralizing the prediction order or direction of several view sequences. For example, if several view sequences to be encoded are given, a center location of the overall arrangement is set to a base view, such that view sequences to be encoded can be hierarchically selected. The end of the overall arrangement or other parts may be set to the base view.
If the number of camera views is denoted by an exponential power of "2", a hierarchical prediction structure between several view sequences may be formed on the basis of the above-mentioned case of the camera views denoted by the exponential power of "2". Otherwise, if the number of camera views is not denoted by the exponential power of "2", virtual views can be used, and the prediction structure may be formed on the basis of the virtual views. If the camera arrangement is indicative of a two- dimensional arrangement, the prediction order may be established by turns in a horizontal or vertical direction.
A parsed bitstream is entropy-decoded by an entropy decoding unit 11, and data such as a coefficient of each macroblock, a motion vector, etc., are extracted. The inverse quantization/inverse transform unit 12 multiplies a received quantization value by a predetermined constant to acquire a transformed coefficient value, and performs an inverse transform of the acquired coefficient value, such that it reconstructs a pixel value. The inter-prediction unit 13 performs an inter-prediction function from decoded samples of the current picture using the reconstructed pixel value.
At the same time, the deblocking filter 15 is ■ applied to each decoded macroblock to reduce the degree of block distortion. The deblocking filter 15 performs a smoothing of the block edge, such that it improves an image quality of the decoded frame. The selection of a filtering process is dependent on a boundary strength and a gradient of image samples arranged in the vicinity of the boundary. The filtered pictures are stored in the decoded picture buffer 16, such that they can be outputted or be used as reference pictures.
The decoded picture buffer 16 stores or outputs pre- coded pictures to perform the inter-prediction function. In this case, frame number information "frame_num" and POC (Picture Order Count) information of the pictures are used to store or output the pre-coded pictures . Pictures of other view may exist in the above-mentioned pre-coded pictures in the case of the MVC technology. Therefore, in order to use the above-mentioned pictures as reference pictures, not only the "frarae_num" and POC information, but also view identifier indicating a picture view may be used as necessary.
The inter-prediction unit 13 performs the inter- prediction using the reference pictures stored in the decoded picture buffer 16. The inter-coded macroblock may be divided into macroblock partitions . Each macroblock partition can be predicted by one or two reference pictures.
The motion compensation unit 17 compensates for a motion of the current block using the information received from the entropy decoding unit 11. The motion compensation unit 17 extracts motion vectors of neighboring blocks of the current block from the video signal, and obtains a motion-vector predictor of the current block. The motion compensation unit 17 compensates for the motion of the current block using a difference value between the motion vector and a predictor extracted from the video signal and the obtained motion-vector predictor. The above-mentioned motion compensation may be performed by only one reference picture, or may also be performed by a plurality of reference pictures . Therefore, if the above-mentioned reference pictures are determined to be pictures of other views different from the current view, the motion compensation may be performed according to a view identifier indicating the other views.
A direct mode is indicative of a coding mode for predicting motion information of the current block on the basis of the motion information of a block which is completely decoded. The above-mentioned direct mode can reduce the number of bits required for encoding the motion information, resulting in the increased compression efficiency.
For example, a temporal direct mode predicts motion information of the current block using a correlation of motion information of a temporal direction. Similar to the temporal direct mode, the decoder can predict the motion information of the current block using a correlation of motion information of a view direction.
If the received bitstream corresponds to a multiview sequence, view sequences may be captured by different cameras respectively, such that a difference in illumination may occur due to internal or external factors of the cameras. In order to reduce potential inefficiency associated with the difference in illumination, an illumination compensation unit 18 performs an illumination compensation function.
In the case of performing illumination compensation function, flag information may be used to indicate whether an illumination compensation at a specific level of a video signal is performed. For example, the illumination compensation unit 18 may perform the illumination compensation function using flag information indicating whether the illumination compensation of a corresponding slice or macroblock is performed. Also, the above- mentioned method for performing the illumination compensation using the above-mentioned flag information may be applied to a variety of macroblock types (e.g., an inter 16x16 mode, a B-skip mode, a direct mode, etc.)
In order to reconstruct the current block when performing the illumination compensation, information of a neighboring block or information of a block in views different from a view of the current block may be used, and an offset value of the current block may also be used.
In this case, the offset value of the current block is indicative of a difference value between an average pixel value of the current block and an average pixel value of a reference block corresponding to the current block. As an example for using the above-mentioned offset value, a predictor of the current-block offset value may be obtained by using the neighboring blocks of the current block, and a residual value between the offset value and the predictor may be used. Therefore, the decoder can reconstruct the offset value of the current block using the residual value and the predictor. In order to obtain the predictor of the current block, information of the neighboring blocks may be used as necessary.
For example, the offset value of the current block can be predicted by using the offset value of a neighboring block. Prior to predicting the current-block offset value, it is determined whether the reference index of the current block is equal to a reference index of the neighboring blocks. According to the determined result, the illumination compensation unit 18 can determine which one of neighboring blocks will be used or which value will be used.
The illumination compensation unit 18 may perform the illumination compensation using a prediction type of the current block. If the current block is predictively encoded by two reference blocks, the illumination compensation unit 18 may obtain an offset value corresponding to each reference block using the offset value of the current block. As described above, the inter-predicted pictures or intra-predicted pictures acquired by the illumination compensation and motion compensation are selected according to a prediction mode, and reconstructs the current picture.
A variety of examples of encoding/decoding method for reconstructing a current picture are described later in this document.
FIG. 2 is a flow chart illustrating a method for encoding a video sequence.
Referring to FIG. 2, an example of a video-sequence encoding method obtains an average pixel value of at least one block from among neighboring blocks of a current block and reference blocks of another view at step S131. Upon receipt of the obtained value, the video-sequence encoding method derives a predicted average pixel value of the current block using at least one mode from among several modes at step S132. The video-sequence encoding method obtains a difference value between the predicted average pixel value and the actual average pixel value of the current block at step S133. The video-sequence encoding method measures individual encoding efficiency of the above-mentioned several modes, and selects an optimum mode from among the several modes at step S134. The above- mentioned optimum mode can be selected in various ways, for example, a method for selecting a minimum difference value from among the obtained difference values, and a method for using an equation indicating the relationship of Rate- Distortion (RD), etc.
In this case, the above-mentioned RD equation recognizes not only the number of encoding bits generated during the encoding of a corresponding block but also a distortion value indicating a difference value associated with an actual image, such that it calculates costs using the number of encoding bits and the distortion value. In more detail, the video-sequence encoding method multiplies the bit number by a Lagrange multiplier determined by a quantization coefficient, and adds the distortion value to the multiplied result, such that it calculates the costs. If the optimum mode is selected, the video-sequence encoding method can encode identification (ID) information indicating the selected mode, and transmit the encoded result. Alternatively, if the optimum mode is selected, the video-sequence encoding method can encode not only the ID information indicating the selected mode but also the difference value obtained by the selected mode, and transmit the encoded result at step S135.
FIG. 3 is a block diagram illustrating a process for deriving a predicted average pixel value of a current block from reference blocks of another view.
Referring to FIG. 3, it is assumed that an average pixel value of the Bc block is mc, an average pixel value of the Br,i block is mr,i, and an average pixel value of the remaining blocks is represented by the above-mentioned block notation. There are a variety of methods for predicting mc information according to information of one or more neighboring blocks. For the convenience of description, it is assumed that the reference frame #1 is used as a candidate reference frame in the case of encoding the Bc block.
A first method for predicting mc information according to information of one or more neighboring blocks is a first mode method (Model) for predicting the mc information on the basis of an average pixel value of a reference block of another view corresponding to the current block. In more detail, the first mode method (Model) is indicative of the method for predicting the mc information using the average pixel value the Br,i block of the reference frame #1. The difference value can be represented by the following equation 1: [Equation 1]
e = mc - mrA
A second method for predicting a difference value between an average pixel value of a current block and an average pixel value of a reference block of another view corresponding to the current block is a second mode method (Mode2) for predicting the difference value on the basis of a difference between average pixel values of each neighboring blocks of the current block and the reference block. In more detail, the second mode method (Mode2) predicts a difference value between an average pixel value of the current block and an average pixel value of the Br/1 block of the reference frame #1 using a difference value in
average pixel values between neighboring blocks ( Bc',Br'l ) .
The difference value can be represented by the following equation 2:
[Equation 2]
e = {mc-mrλ)-{mc ι-mr ι λ)
A third method for predicting a difference value between an average pixel value of a current block and an average pixel value of a reference block of another view corresponding to the current block is a third mode method
(Mode3) for predicting the difference value using a difference between an average pixel value of a neighboring block of the current block and an average pixel value of the reference block. In more detail, the third mode method
(Mode3) predicts the mc information on the basis of a difference between an average pixel value of the
neighboring block Bc' and an average pixel value of the BrJ
block of the reference frame #1. In this case, the difference value can be represented by the following equation 3 :
[Equation 3]
Figure imgf000045_0001
In the case of encoding a neighboring block of the current block by using the neighboring blocks of the reference block of another view, there is a fourth mode method (Mode4) for predicting the mc information on the basis of predicted average pixel values of the neighboring
blocks of the current block. In other words, if the Bc'
block is pre-encoded by referring to the Br'2 block of the
reference frame #2, a difference value between the average
pixel value of the current block ( B. ) and a reference block ( B1. , ) corresponding to the current block can be
predicted by a difference value between the average pixel
value of the neighboring block of the current block ( Bc ] )
and an average pixel value of neighboring block of another
view reference block (Bl2).
In this case, the difference value can be represented by the following equation 4 : [Equation 4]
Figure imgf000046_0001
In the case of using the neighboring-block information using the above-mentioned Mode2, Mode3, and Mode4 methods, although the above-mentioned Mode2, Mode3, and Mode4 methods have disclosed that only one information of the next upper-block is exemplarily used, it should be noted that the combination of information of several neighboring blocks surrounding the current block may also be used as an example.
FIG. 4 is a detailed block diagram illustrating a process for deriving a predicted average pixel value of a current block from reference blocks of other views.
In more detail, FIG. 4 shows a current block, pre- encoded blocks, each of which shares a boundary with the current block, and other blocks, each of which shares a boundary with the reference block. In this case, the Mode2-method equation, the Mode3-method equation, and the Mode4-method equation can be represented by the following equation 5 :
[Equation 5]
Figure imgf000047_0001
In the above-mentioned Mode4 equation, m\,k indicates
an average pixel value of a. reference block of the Bc' block
on the condition that the reference block is located at the reference frame #k.
In Equation 5, W1 indicates a weighted coefficient.
The neighboring blocks used for prediction are not limited to blocks sharing a boundary, and may also include other blocks adjacent to the above-mentioned neighboring blocks as necessary. Otherwise, the above-mentioned neighboring blocks may also employ only some parts of the other blocks. The scope of the above-mentioned neighboring blocks may be
adjusted by the w, . In this way, the difference value (e)
is quantized and entropy-encoded, such that the entropy- encoded information is transmitted to the decoding unit. The reference frames of the above-mentioned Model, Mode2, Mode3, and Mode4 methods are determined to be optimum frames in consideration of rate and distortion factors after calculating several steps to an actual bitstream stage. There are a variety of methods for selecting the optimum mode, for example, a method for selecting a specific mode of a minimum difference value from among the obtained difference values, and a method for using the RD relationship.
The above-mentioned RD-relationship method calculates actual bitstreams of individual modes, and selects an optimum mode in consideration of the rate and the distortion. In the case of calculating a block residual value, the above-mentioned RD-relationship method deducts an average pixel value of each block from the current block, deducts the average pixel value of each block from the reference block, and calculates a difference value between the deducted results of the current and reference blocks, as represented by the following equation 6: [ Equation 6 ]
Figure imgf000049_0001
In Equation 6, ΔxΔy is indicative of a disparity
vector, and / is a pixel value. If a value predicted by information of a neighboring block and a difference value are quantized, and the quantized resultant values of the predicted value and the difference value are reconstructed, and the reconstructed resultant values are added, the added
result is denoted by mc of Equation 6. In this case, the
value of mc is adapted to obtain the same values from the
encoding unit and the decoding unit. mr is indicative of an
average pixel value of a reference block. In the case of
the decoded image, the encoding unit has the same mr as
that of the decoding unit. Indeed, the reference block is searched for in a time domain, and an optimum block is searched for in a space-time domain. Therefore, ID information indicating whether an illumination compensation will be used is set to "0" or "1" in association with individual frames and blocks, and the resultant ID information is entropy-encoded.
If the optimum mode is selected, it is possible to encode only the selected mode, such that the encoded result of the selected mode may be transmitted to the decoding unit. In addition to the encoded result of the selected mode, a difference value obtained by the selected mode can also be encoded and transmitted. The selected mode information is represented by index types, and can also be predicted by neighboring-mode information. In addition, a difference value between the index of the currently- selected mode and the index of the predicted mode can also be encoded and transmitted.
All of the above-mentioned modes may be considered, some of the above-mentioned modes may be selected, or only one of the above-mentioned modes may also be selected as necessary. In the case of using a single method from among all available methods, there is no need to separately encode the mode index.
In the case of obtaining an average pixel value and deriving a predicted average pixel value, pre-decoded pixel values may be applied to current blocks of a reference frame and a target frame to be encoded.
Basically, pre-decoded values of left-side pixels and pre-decoded values of upper-side pixels are used to predict an average pixel value of the current block. In the case of encoding an actual video sequence, the video sequence is encoded on the basis of a macroblock. The 16x16 macroblock is divided into 16x8 blocks, 8x16 blocks, and 8x8 blocks, and is then decoded. The 8x8 blocks may also be divided into 8x4 blocks, 4x8 blocks, and 4x4 blocks. There are a variety of methods for predicting an average pixel value of sub-blocks on the basis of a single macroblock.
FIG. 5 is a conceptual diagram illustrating a 16x16 macroblock for explaining usages of pre-decoded pixel values located at left- and upper- parts of an entire block in the case of deriving an average pixel value and a predicted average pixel value of a current block.
Referring to FIG. 5, the 16x16 macroblock can use all the pixel values of the left- and upper- parts. Therefore, in the case of predicting an average pixel value of the current block, an average pixel value of pixels
(hl~hl6) of the upper part and pixels (vl~vl6) of the
left part is calculated, and an average pixel value of the current block is predicted by the calculated average pixel
value of the pixels (vl~vl6, hl~hl6). In this case, the
average pixel value of the 16x16 block (denoted by "B16xl6") can be represented by the following equation 7: [Equation 7] 16 16
32
FIG. 6A is a conceptual diagram illustrating a 16x8 macroblock for explaining usages of all the pixels enclosing divided blocks in the case of deriving an average pixel value and a predicted average pixel value of the divided blocks. FIG. 6B is a conceptual diagram illustrating a 16x8 macroblock for explaining usages of only pixels enclosing divided blocks in the case of deriving an average pixel value and a predicted average pixel value of the divided blocks.
In FIG. 6A, in the case of using all the pixels enclosing the divided blocks, an average value of the B16x8_0 block and the B16x8__l block can be represented by the following equation 8 : [Equation 8]
Figure imgf000052_0001
32
In FIG. 6B, in the case of using all the pixels enclosing the divided blocks, an average value of the B16x8_0 block can be represented by the following equation 9, and an average value of the B16x8_l block can be represented by the following equation 10: [Equation 9]
Figure imgf000053_0001
J=I f=l 24
[Equation 10 ]
Figure imgf000053_0002
J=I i=9
24
In the above-mentioned cases of FIGS. 6A~6B, the
value of ho located at the corner of the macroblock may also be added to the calculation result as necessary. In this case, an average pixel value of the B16x8_0 block of FIG. 6A can be represented by the following equation 11, and the average pixel value of the B16x8_0 of FIG. 6B can be represented by the following equation 12: [Equation 11]
Figure imgf000053_0003
33
[Equation 12 ]
Figure imgf000054_0001
25
In the above-mentioned cases of FIGS. 6A~6B, the
values of h0 and v8 located at the corners of the macroblock may also be added to the calculation result as necessary. In this case, an average pixel value of the B16x8_l block of FIG. 6A can be represented by the following equation 13, and the average pixel value of the B16x8_l of FIG. 6B can be represented by the following equation 14: [Equation 13]
Figure imgf000054_0002
33
[Equation 14 ]
Figure imgf000054_0003
Z=O z=8
25
FIG. 7A is a conceptual diagram illustrating a 8x16 macroblock for explaining usages of all the pixels enclosing divided blocks in the case of deriving an average pixel value and a predicted average pixel value of the divided blocks. FIG. 7B is a conceptual diagram illustrating a 8x16 macroblock for explaining usages of only pixels enclosing divided blocks in the case of deriving an average pixel value and a predicted average pixel value of the divided blocks . The method for deriving an average pixel value of the divided blocks is the same as
that of FIGS. 6A-6B.
FIG. 8A is a conceptual diagram illustrating a 8x8 macroblock for explaining usages of all the pixels enclosing divided blocks in the case of deriving an average pixel value and a predicted average pixel value of the divided blocks. FIG. 8B is a conceptual diagram illustrating a 8x8 macroblock for explaining usages of only pixels enclosing divided blocks in the case of deriving an average pixel value and a predicted average pixel value of the divided blocks. The method for deriving an average pixel value of the divided blocks is the same as that of
FIGS. 6A-6B.
The 8x8 block can be divided into a plurality of sub-blocks .
An average pixel value of a corresponding block of a current block of a current frame to be encoded is predicted,
such that the predicted average pixel value is set to mc . An average pixel value of a corresponding block of the reference frame is predicted, such that the predicted
average pixel value is set to mr .
Each predicted average pixel value is deducted from all pixels of each block, and a difference value between the predicted pixel value using the reference block and a pixel value of the current block can be calculated by the following equation 15: [Equation 15]
ΣΣ Ie(i,j)-mc-VrV+*xJ+Δy)-mr)
In Equation 15, (Λx,Ay) is indicative of a disparity-
vector, and I is a pixel value. A reference block having a minimum block residual value is selected as an illumination-compensated optimum block. In this case, the disparity vector is denoted by (Ax,Ay) . Indeed, a system
compares the above-mentioned illumination-compensated case with another case in which the illumination is not compensated, and selects a superior one of the two cases. As a modified example of the above-mentioned scheme, an average pixel value of the reference block is not predicted by pixel values of neighboring blocks, and is directly calculated by an average pixel value of all pixels contained in an actual block.
As another modified example of the above-mentioned scheme, the number of left- and upper-part pixels may be increased. In more detail, pixels of two or more neighboring layers of a current layer may be used instead of pixels of only one layer next to a current layer.
The decoding unit determines whether to perform an illumination compensation of a corresponding block using the ID information. If the illumination compensation is performed, the decoding unit calculates a decoded value of the difference value (e) , and obtains a predicted value according to an above-mentioned prediction method. The decoded value of the difference value (e) is added to the
predicted value, such that the value of mc (= mc + e) can be
decoded. The value of mr—mc is deducted from the reference block, which is prediction block so called predictor for the current block, and the deducted result is added to the decoded value of the residual block, such that the value of the current block can be finally obtained. The current block can be reconstructed as follow:
B = prediction block + residual block + {mc-mr + e) , where B is the value of the current block, reference block is the predictor for the current block, m.-m,- is a predicted difference of average pixel values, that is the predicted offset value of illumination compensation for the current block, and e is the difference value. The decoding unit obtains the difference between a offset value of illumination compensation of the current block and a predicted difference, and can reconstruct the offset value of illumination compensation of the current block using the obtained residual block value and the predicted difference. FIG. 9 is a diagram illustrating a process for obtaining an offset value of a current block.
The illumination compensation may be performed during the motion estimation. When it compares the current block with the reference block, a difference in illumination between two blocks is considered. New motion estimation and new motion compensation are used to compensate for the illumination difference. A new SAD (Sum of Absolute Differences) can be represented by the following equations 16 and 17: [Equation 16]
SAD= Σ Yϊlc{^y)-Mc)-{Ir{x+^y+^y)-Mr)\ x=m y=n
Figure imgf000058_0001
y=n [Equation 17]
A/+m-l Λ'+M-1
A/ +W-I yV+H-1
M1. = ∑ ∑Ir(x+ Ax,y + Δy) x=m v=n
With reference to Equations 16 and 17, Mc is indicative of an average pixel value of the current block, and Mr is indicative of an average pixel value of the
reference block. Ic(x,y) is indicative of a pixel value at a
specific coordinate (x,y) of the current block, and
Ir(x+ Ax,y+ Ay) is indicative of a pixel value at a motion
vector (Ax,Ay) of the reference block. The motion
estimation is performed on the basis of the new SAD denoted by Equation 16, such that a difference value between an average pixel value of the current block and an average pixel value of the reference block can be obtained. The difference value in average pixel value between the current block and the reference block is referred to as an offset value (IC_offset) .
If the motion estimation applying for the illumination compensation is performed, the offset value and the motion vector are obtained. The illumination compensation can be performed by the following equation 18 using the offset value and the motion vector: [Equation 18]
R(x,y) = Ic(x,y)-I,.(x+Ax,y+Ay)-(Mc-Mr)
With reference to Equation 18, R(x,y) is indicative
of an illumination-compensated residual value. The offset value (IC_offset = Mc-Mr) is transmitted to the decoding unit. The illumination compensation of the decoding unit can be performed by the following equation 19:
[Equation 19]
fc(x,y) = Ir(x+Ax,y+Δy)+R'(x,y)+(MC-Mr)
With reference to Equation 19, R(x,y) is indicative
of an reconstructed and illumination-compensated residual
value, and Ic(x,y) is indicative of a pixel value of the
current block. In order to reconstruct the current block, the offset value is transmitted to the decoding unit, and the offset value can be predicted by data of the neighboring blocks. In order to further reduce the number of bits for coding the offset value, a difference value (R10 offset) between the current-block offset value (IC_offset) and the neighboring-block offset value (IC_offset_pred) can be transmitted to the decoding unit 50, as denoted by the following equation 20: [Equation 20 ]
Ric_oθse, = IC_ offset - IC_offset _pred
FIG. 10 is a flow chart illustrating a process for performing for an illumination compensation of a current block.
Referring to FIG. 10, if an illumination compensation flag of a current block is set to "0", the illumination compensation of the current block is not performed. Otherwise, if the illumination compensation flag of the current block is set to "1", a process for reconstructing the offset value of the current block is performed. In the case of obtaining a predictor of the current block, information of the neighboring block can be employed. It is determined whether a reference index of the current block is equal to a reference index of the neighboring block at step S210. A predictor for performing the illumination compensation of the current block is obtained on the basis of the determined result at step S211. An offset value of the current block is reconstructed by using the obtained predictor at step S212. In this case, the step S210 for determining whether the reference index of the current block is equal to that of the neighboring block and the step S211 for obtaining the predictor on the basis of the determined result will hereinafter be described with reference to FIG. 11.
FIG. 11 is a flow chart illustrating a method for obtaining a predictor by determining whether a reference index of a current block is equal to a reference index of a neighboring block.
Referring to FIG. 11, in order to perform an illumination compensation, the decoding unit extracts a variety of information from a video signal, for example, flag information and offset values of neighboring blocks of the current block, and reference indexes of reference blocks of the current and neighboring blocks, such that the decoding unit can obtain the predictor of the current block using the extracted information. The decoding unit obtains a residual value between the offset value of the current block and the predictor, and can reconstruct the offset value of the current block using the obtained residual value and the predictor.
In the case of obtaining the predictor of the current block, information of the neighboring block can be employed. For example, the offset value of the current block can be predicted by the offset value of the neighboring block. Prior to predicting the offset value of the current block, it can be determined whether the reference index of the current block is equal to that of the neighboring block, such that it can be determined which one of values or which one of neighboring blocks will be used by referring to the determined result. Also, it is determined whether flag information of the neighboring block is set to "true", such that it can be determined whether the neighboring block will be used by referring to the determined result.
According to a first example, it is determined whether the neighboring block having the same reference index as that of the current block exists at step S220. If it is determined that only one neighboring block having the same reference index as that of the current block exists, an offset value of the neighboring block having the same reference index is assigned to the predictor of the current block at step S221. If it is determined that two neighboring blocks, each of which has the same reference index as that of the current block, exist at step S220, an average value of the offset values of the two neighboring blocks is assigned to the predictor of the current block at step S222. If it is determined that three neighboring blocks, each of which has the same reference index as that of the current block, exist at step S220, a median value of the offset values of the three neighboring blocks is assigned to the predictor of the current block at step S223. If it is determined that there is no neighboring block having the same reference index as that of the current block according to the determined result at step S220, the predictor of the current block is set to "0" at step S224. If required, the step S220 for determining whether the reference index of the current block is equal to that of the neighboring block may further include another step for determining whether a flag of the neighboring block is set to "1". According to a second example, it is determined whether the neighboring block has the same reference index as that of the current block, and it is determined whether a flag of the neighboring block is set to "1". If it is determined that the neighboring block has the same reference index as that of the current block, and has the flag of "1", an offset value of the neighboring block may be set to the predictor of the current block. In this case, a plurality of neighboring blocks may be checked in the order of a left neighboring block -> an upper neighboring block -^ a right-upper neighboring block -> a left-upper neighboring block. If required, the neighboring blocks may also be checked in the order of the upper neighboring block
-> the left neighboring block -> the right-upper neighboring block -> the left-upper neighboring block. If there is no neighboring block capable of satisfying the two conditions, and flags of the three neighboring blocks
(i.e., the left neighboring block, the upper neighboring block, and the right-upper (or left-upper) neighboring block) are set to "1", respectively, the median value of the offset values of the three blocks is set to the predictor. Otherwise, the predictor of the current block may be set to "0".
FIG. 12 is a flow chart illustrating a method for performing for an illumination compensation on the basis of a prediction type of a current block.
Referring to FIG. 12, the neighboring block acting as a reference block may be changed according to a prediction type of the current block. For example, if the current block has the same shape as that of the neighboring block, the current block is predicted by a median value of the neighboring blocks. Otherwise, if the shape of the current block is different from that of the neighboring block, another method will be employed. For example, if a block located at the left side of the current block is divided into several sub-blocks, the uppermost sub-block from among the sub-blocks is used for the prediction. Also, if a block located at an upper part of the current block is divided into several sub-blocks, the leftmost sub-block is used for the prediction. In this case, a prediction value may be changed according to the prediction type of the current block. Therefore, the example of FIG. 12 determines a neighboring block to be referred by the prediction type of the current block at step S231. It is determined whether the reference index of the determined neighboring block is equal to a reference index of the current block at step S232. The step S232 for determining whether the reference index of the neighboring block is equal to that of the current block may further include another step for determining whether a flag of the neighboring block is set to "1". The predictor for performing an illumination compensation of the current block can be obtained on the basis of the determined result at step S233. The offset value of the current block is reconstructed by the obtained predictor, such that the illumination compensation can be performed at step S234. In this case, the process for performing the step S233 by referring to the result of step S232 will hereinafter be described in detail, and a detailed description thereof will be similar to that of FIG. 11.
For example, if the prediction type of the current block indicates that the prediction is performed by using a neighboring block located at the left side of the current block, it is determined whether the reference index of the left-side neighboring block is equal to that of the current block. If the reference index of the current block is equal to that of the left-side neighboring block, an offset value of the left-side neighboring block is assigned to the predictor of 'the current block. Also, if the prediction type of the current block indicates that the prediction is performed by referring to the left- and upper- neighboring blocks of the current block, or if the prediction is performed by referring to three neighboring blocks (i.e., the left neighboring block, the upper neighboring block, and the right-upper neighboring block) , the individual cases will be applied similarly as a method of FIG. 11.
FIG. 13 is a flow chart illustrating a method for performing for an illumination compensation using flag information indicating whether the illumination compensation of a block is performed.
Referring to FIG. 13, flag information (IC_flag) indicating whether an illumination compensation of the current block is performed may also be used to reconstruct the offset value of the current block. In addition, the predictor may also be obtained using both the method for checking the reference index of FIG. 11 and the method for predicting flag information. Firstly, it is determined whether a neighboring block having the same reference index as that of the current block exists at step S241. A predictor for performing an illumination compensation of the current block is obtained by the determined result at step S242. In this case, a process for determining whether the flag of the neighboring block is "1" may also be included in the step S242. The flag information of the current block is predicted on the basis of the determined result at step S243. An offset value of the current block is reconstructed by using the obtained predictor and the predicted flag information, such that the illumination compensation can be performed at step S244. In this case, the step S242 may be applied similarly as a method of FIG. 11, and the step S243 will hereinafter be described with reference to FIG. 14.
FIG. 14 is a flow chart illustrating a method for predicting flag information of a current block by determining whether a reference index of the current block is equal to a reference index of a neighboring block. Referring to FIG. 14, it is determined whether the neighboring block having the same reference index as that of the current block exists at step S250. If it is determined that only one neighboring block having the same reference index as that of the current block exists, flag information of the current block is predicted by flag information of the neighboring block having the same reference index at step S251. If it is determined that two neighboring blocks, each of which has the same reference index as that of the current block, exist at step S250, flag information of the current block is predicted by any one of flag information of the two neighboring blocks having the same reference index at step S252.
If it is determined that three neighboring blocks, each of which has the same reference index as that of the current block, exist at step S250, the flag information of the current block is predicted by a median value of the flag information of the three neighboring blocks at step S253. Also, if there is no neighboring block having the same reference index as that of the current block according to the determined result of step S250, the flag information of the current block is not predicted at step S254.
FIG. 15 is a flow chart illustrating a method for performing an illumination compensation when a current block is predictively coded by two or more reference blocks.
Referring to FIG. 15, during performing the illumination compensation, if the current block is predictively coded by using two reference blocks, the decoding unit cannot directly recognize an offset value corresponding to each reference block, because it uses an average pixel value of the two reference blocks when obtaining the offset value of the current block. Therefore, in one example, an offset value corresponding to each reference block is obtained, resulting in the implementation of correct prediction. The offset value of the current block is reconstructed by using the predictor of the current block and the residual value at step S261. If the current block is predictively encoded by using two reference blocks, an offset value corresponding to each reference is obtained by the offset value at step S262, as denoted by the following equation 21: [Equation 21]
Figure imgf000070_0001
IC_offsetLO — mc —mr ,
Figure imgf000070_0002
-1)x mr , +W2 xmrP
IC__offsetLl = mc-mr2 =IC_offset+ w] xmrl + (W2 -1)xmr,
In Equation 21, mc is an average pixel value of the
current block. πιr , and mr2 are indicative of an average
pixel values of reference blocks, respectively. W1 and
W2 are indicative of a weighted coefficients for a bi-
predictive coding process, respectively. ° In one example of the illumination compensation method, the system independently obtains an accurate offset value corresponding to each reference block, such that it can more correctly perform the predictive coding process. In the case of reconstructing the offset value of the current block at step S262, the system adds the reconstructed residual value and the predictor value, such that it obtains an offset value. In this case, the predictor of a reference picture of ListO and the predictor of a reference picture of Listl are obtained repectively and combined, such that the system can obtain a predictor used for reconstructing the offset value of the current block.
According to another example, the system can also be applied to skip-macroblock. In this case, the prediction is performed to obtain an information for the illumination- compensation. A value predicted by the neighboring block block is used as flag information indicating whether the illumination compensation is performed. An offset value predicted by the neighboring block may be used as the offset value of the current block. For example, if flag information is set to "true", the offset value is added to a reference block. In the case of a macroblock to which a P-skip mode is applied, the prediction is performed by using flags and offset values of the left- and upper- neighboring blocks, such that flag and offset values of the macroblock can be obtained. If only one block has the flag of "1", a flag and an offset value of the current block may be set to the flag and the offset value of the block, respectively. If two blocks have the flag of "1", the flag of the current block is set to "1", and the offset value of the current block is set to an average offset value of the two neighboring blocks.
According to another example, the system can also be applied to a direct mode, for example, temporal direct mode, B-skip mode, etc. In this case, the prediction is performed to obtain information of the illumination- compensation. Each predictor can be obtained by using the variable method for predicting the flag and the offset. This predictor may be set to an actual flag and an actual offset value of the current block. If each block has a pair of flags and offset information, a prediction value for each block can be obtained. In this case, if there are two reference blocks and the reference indexes of the two reference blocks are checked, it is determined whether the reference index of the current block is equal to that of the neighboring block. Also, if each reference block includes a unique offset value, first predicted flag information, a first predicted offset value, second predicted flag information, and a second predicted offset value can be obtained. In this case, a value predicted by the neighboring block may be used as the flag information. The offset values of the two reference blocks may be used as the first predicted offset value and the second predicted offset value, respectively. In this case, the offset value of the current block may be set to an average offset value of individual reference blocks.
In the direct mode or the skip macroblock mode, the system may encode/decode the flag information indicating whether the direct mode or the skip-macroblock mode is applied to the current block. In more detail, an offset value is added or not according to the flag value. A residual value between the offset value and the predicted offset value may also be encoded/decoded. In this case, desired data can be more correctly reconstructed, and an optimum mode may be selected in consideration of a RD (Rate-Distortion) - relationship. If a reference picture cannot be used for the prediction process, i.e., if a reference picture number is less than "1", the flag information or predicted flag information may be set to "false", and the offset value or the predicted offset value may also be set to "0".
According to another example, the system can also be applied to the entropy-coding process. In association with the flag information, three context models may be used according to flag values of the neighboring blocks (e.g., blocks located at the left- and upper- parts of the current block) . If it is determined that the flag value is set to "true", the value of "1" occurs. If it is determined that the flag value is set to "false", the value of "0" occurs. If the two values "1" and "0" of the two cases are added, three cases can be obtained. The flag information is encoded/decoded by using the three context models. A transform-coefficient level coding method can be used for the predictive residual value of the offset values. In other words, data binarization is performed by UEGO, a single context model can be applied to a first bin value, and another context mode is applied to the remaining bin values of a unary prefix part A sign bit is encoded/decoded by a bypass mode. According to another example of the flag information, two contexts may be considered according to a predicted flag values, such that the encoding/decoding process can be performed.
FIG. 16 is a flow chart illustrating a method for performing illumination compensation using not only flag information indicating whether illumination compensation of a current block is performed, but also an offset value of the current block.
Referring to FIG. 16, in order to perform illumination compensation, the decoding unit extracts a variety of information from a video signal, for example, flag information and offset values of the current and neighboring blocks of the current block, and index information of reference blocks of the current and neighboring blocks, such that the decoding unit can obtain the predictor of the current block using the above- mentioned extracted information. The decoding unit 50 obtains a residual value between the offset value of the current block and the predictor, and can reconstruct the offset value of the current block using the obtained residual value and the predictor. In the case of reconstructing the offset value of the current block, flag information (IC_flag) indicating whether the illumination compensation of the current block is performed may be used.
The decoding unit obtains flag information indicating whether the illumination compensation of the current block is performed at step S271. If the illumination compensation is performed according to the above-mentioned flag information (IC_flag) , the offset value of the current block indicating a difference in average pixel value between the current block and the reference block can be reconstructed at step S272. In this way, the above-mentioned illumination compensation technology encodes a difference value in average pixel value between blocks of different pictures. If a corresponding block is contained in the P slice when the flag indicating whether the illumination compensation is applied to each block, single flag information and a single offset value are encoded/decoded. However, if the corresponding block is contained in the B slice, a variety of methods can be made available, and a detailed description thereof will hereinafter be described with
reference to FIGS. 17A-17B.
FIGS. 17A-17B are diagrams illustrating a method for
performing illumination compensation using flag information and an offset value in association with blocks of P and B slices .
Referring to FIG. 17A, "C" is indicative of a current block, "N" is indicative of a neighboring block of the current block (C) , "R" is indicative of a reference block of the current block (C) , "S" is indicative of a reference block of the neighboring block (N) of the current block (C) , and "mc" is indicative of an average pixel value of the current block (C) , "mr" is indicative of an average pixel value of the reference block of the current block (C) If the offset value of the current block (C) is denoted by "IC_offset", the "IC_offset" information can be denoted by "IC_offset = mc - mr" .
In this way, if the offset value of the neighboring block (S) is denoted by "IC_offset_pred", the encoding unit can transmit the residual value (RIC offset) between the offset value (IC_offset) of the current block and the offset value (IC__offset_pred) of the neighboring block to a decoding unit, such that it can reconstruct the offset value "IC_offset" of the current block (C) . In this case, the "Ric offset" information can also be represented by the above-mentioned Equation 20.
In' the case of generating the predictor of the current block on the basis of flag information or offset value of the neighboring block, a variety of methods can be made available. For example, information of only one neighboring block may be employed, or information of two or more neighboring blocks may also be employed. In the case of employing the information of two or more neighboring blocks, an average value or a median value may be employed. In this way, if the current block is predictively encoded by a single reference block, the illumination compensation can be performed using a single offset value and single flag information. However, if the corresponding block is contained in the B slice, i.e., if the current block is predictively encoded by two or more reference blocks, a variety of methods can be made available. For example, as shown in FlG. 17B, it is assumed that "C" is indicative of a current block, "N" is indicative of a neighboring block of the current block (C) , "RO" is indicative of a reference block located at a reference picture (1) of List 0 referred by the current block, "SO" is indicative of a reference block located at the reference picture (1) of List 0 referred by the neighboring block, "Rl" is indicative of a reference block located at a reference picture (3) of List 1 referred by the current block, and "Sl" is indicative of a reference block located at the reference picture (3) of List 1 referred by the neighboring block. In this case, the flag information and the offset value of the current block are associated with each reference block, such that each reference block includes two values. Therefore, at least one of the flag information and the offset value can be employed respectively.
According to a first example, a predictor of the current block can be obtained by combining information of two reference blocks via the motion compensation. In this case, single flag information indicates whether the illumination compensation of the current block is performed. If the flag information is determined to be "true", a single offset value is obtained from the current block and the predictor, such that the encoding/decoding processes can be performed.
According to a second example, in the motion ■compensation process, it is determined whether the illumination compensation will be applied to each of two reference blocks. Flag information is assigned to each of the two reference blocks, and a single offset value obtained by using the above-mentioned flag information may be encoded or decoded. In this case, it should be noted that two flag information may be used on the basis of the reference block, and a single offset value may be used on the basis of the current block.
According to a third example, single flag information may indicate whether the illumination compensation will be applied to a corresponding block on the basis of the current block. Individual offset values can be encoded/decoded • for two reference blocks. If the illumination compensation is not applied to any one of the reference blocks during the encoding process, a corresponding offset value is set to "0". In this case, single flag information may be used on the basis of the current block, and two offset values may be used on the basis of the reference block.
According to a fourth example, the flag information and the offset value can be encoded/decoded for individual reference blocks. In this case, two flags and two offset values can be used on the bass of the reference block.
According to the above-mentioned first to fourth examples, the offset value is not encoded without any change, and is predicted by an offset value of the neighboring block, such that its residual value is encoded.
FIG. 18 is a flow chart illustrating a method for performing an illumination compensation when a current block is predictively encoded by two or more reference blocks.
Referring to FIG. 18, in order to perform the illumination compensation on the condition that the current block is contained in the B slice, flag information and offset values of the neighboring blocks of the current block are extracted from the video signal, and index information of corresponding reference blocks of the current and neighboring blocks are extracted, such that the predictor of the current block can be obtained by using the extracted information. The decoding unit obtains a residual value between the offset value of the current block and the predictor, and can reconstruct the offset value of the current block using the obtained residual value and the predictor. In the case of reconstructing the offset value of the current block, flag information
(IC_flag) indicating whether the illumination compensation of the current block is performed may be used as necessary.
The decoding unit obtains flag information indicating whether the illumination compensation of the current block is performed at step S291. If the illumination compensation is performed according to the above-mentioned flag information (IC_flag), the offset value of the current block indicating a difference in average pixel value between the current block and the reference block can be reconstructed at step S292.
However, if the current block is predictively encoded by two reference blocks, a decoder cannot directly recognize an offset value corresponding to each reference block, because it uses an average pixel value of two reference blocks when obtaining the offset value of the current block. Therefore, according to a first example, an offset value corresponding to each reference is obtained, resulting in the implementation of correct prediction. Therefore, if the current block is predictively encoded by two reference blocks, an offset value corresponding to each reference can be obtained by using the above-mentioned offset value at step S293, as denoted by the following equation 22 : [Equation 22]
Figure imgf000082_0001
Figure imgf000082_0002
=IC_offset + (w, -1)x mr , + W2 x m,.;2
/C_offsetLl = mc -mr 2 = /C_o//Se^+ w, xmr>1 + (w2 -1)xmr2
In Equation 22, mc is an average pixel value of the
current block. mrl and mr2 are indicative of average pixel
values of reference blocks, respectively. W1 and W2 are
indicative of weighted coefficients for a bi-predictive coding process, respectively.
In the case of performing the illumination compensation using the above-mentioned method, the system independently obtains an accurate offset value corresponding to each reference block, such that it can more correctly perform the predictive coding process. In the case of reconstructing the offset value of the current block, the system adds the reconstructed residual value and the predictor value, such that it obtains the offset value. In this case, the predictor of List 0 and the predictor of
List 1 are obtained and combined, such that the system can obtain a predictor value used for reconstructing the offset value of the current block.
FIG. 19 is a flow chart illustrating a method for performing an illumination compensation using flag information indicating whether the illumination compensation of a current block is performed.
The illumination compensation technology is adapted to compensate for an illumination difference or a difference in color. If the scope of the illumination compensation technology is extended, the extended illumination compensation technology may also be applied between obtained sequences captured by the same camera. The illumination compensation technology can prevent the difference in illumination or color from greatly affecting the motion estimation. However, indeed, the encoding process employs flag information indicating whether the illumination compensation is performed. The application scope of the illumination compensation may be extended to a sequence, a view, a GOP (Group Of Pictures) , a picture, a slice, a macroblock, and a sub-block, etc.
If the illumination compensation technology is applied to a small-sized area, a local area may also be controlled, however, it should be noted that a large number of bits used for the flag information are consumed. The illumination compensation technology may not be required. Therefore, a flag bit indicating whether the illumination compensation is assigned to individual areas, such that the system can effectively use the illumination compensation technology. The system obtains flag information capable of allowing a specific level of the video signal to be illumination-compensated at step S201.
For example, the following flag information may be assigned to individual areas. "seq_IC_flag" information is assigned to a sequence level, "view__IC_flag" information is assigned to a view level, "GOP_IC_flag" information is assigned to a GOP level, "pic_IC_flag" information is assigned to a picture level, "slice_IC_flag" information is assigned to a slice level, "mb_IC_flag" information is assigned to a macroblock level, and Λ'blk_IC_flag" information is assigned to a block level. A detailed description of the above-mentioned flag information will be
described with reference to FIGS. 20A-20C. A specific
level of the video signal in which the illumination compensation is performed by the flag information can be decoded at step S302.
FIGS. 20A-20C are conceptual diagrams illustrating
the scope of flag information indicating whether illumination compensation of a current block is performed. Referring to FIGS. 20A~20c, the flag information
indicating whether the illumination compensation is performed can hierarchically be classified. For example,
as can be seen from FIGS. 20A-20C, "seq_IC_flag"
information 311 is assigned to a sequence level, NView_IC_flag" information 312 is assigned to a view level, "GOP_IC_flag" information 313 is assigned to a GOP level, "pic_IC_flag" information 314 is assigned to a picture level, "slice_IC_flag" information 315 is assigned to a slice level, "mb_IC_flag" information 316 is assigned to a macroblock level, and "blk_IC_flag" information 317 is assigned to a block level.
In this case, each flag is composed of 1 bit. The number of the above-mentioned flags may be set to at least one. The above-mentioned sequence/view/picture/slice-level flags may be located at a corresponding parameter set or header, or may also be located another parameter set. For example, the "seq_IC_flag" information 311 may be located at a sequence parameter set, the "view_IC_flag" information 312 may be located at the view parameter set, the Mpic__IC_flag" information 314 may be located at the picture parameter set, and the "slice_IC_flag" information 315 may be located at the slice header.
If two or more flags exist, specific information indicating whether the illumination compensation of an upper level is performed may control whether the illumination compensation of a lower level is performed. In other words, if each flag bit value is set to "1", the illumination compensation technology may be applied to a lower level.
For example, if the "pic_IC_flag" information is set to "1", the λNslice__IC_flag" information of each slice contained in a corresponding picture may be set to "1" or w0", the "mb_IC_flag" information of each macroblock may be set to "1" or "0", or the "blk_IC_flag" information of each block may be set to "1" or "0". If the "seq_IC_flag" information is set to NΛ1" on the condition that a view parameter set exists, the "view_IC_flag" value of each view may be set to "1" or "0". Otherwise, if the "view_IC_flag" information is set to "1", a flag bit value of GOP, picture, slice, macroblock, or block of a corresponding view may be set to "1" or xx0", as shown in FIG. 2OA. Needless to say, the above-mentioned flag bit value of GOP, picture, slice, macroblock, or block of the corresponding view may not be set to "1" or u0" as necessary. If the above-mentioned flag bit value of GOP, picture, slice, macroblock, or block of the corresponding view may not be set to "1" or "0", this indicates that the GOP flag, the picture flag, the slice flag, the macroblock flag, or the block flag is not controlled by the view flag information, as shown in FIG 2OB.
If the flag bit value of an upper scope is set to "0", the flag bit values of a lower scope are automatically set to "0". For example, if the xxseq_IC_flag" information is set to "0", this indicates that the illumination compensation technology is not applied to a corresponding sequence. Therefore, the xxview_IC_flag" information is set to "0", the xxGOP_IC_flag" information is set to "0", the "pic_IC_flag" information is set to "0", the "slice_IC_flag" information is set to xx0", the xxmb_IC_flag" information is set to "0", and the xxblk_IC_flag" information is set to "0". If required, only one mb_IC_flag" information or only one xxblk__IC_flag" information may be employed according to a specific implementation methods of the illumination compensation technology. If required, the "view_IC_flag" information may be employed when the view parameter set is newly applied to the multiview video coding. The offset value of the current block may be additionally encoded/decoded according to a flag bit value of the macroblock or sub- block acting as the lowest-level unit.
As can be seen from FIG. 2OC, the flag indicating the IC technique application may also be applied to both the slice level and macroblock level. For example, if the "slice_IC_flag" information is set to "0", this indicates that the IC technique is not applied to a corresponding slice. If the "slice__IC_flag" information is set to "1", this indicates that the IC technique is applied to a corresponding slice. In this case, if the "mb_IC_flag" information is set to λXl", "IC__offset" information of a corresponding macroblock is reconstructed. If the "mb_IC_flag" information is set to "0", this indicates that the IC technique is not applied to a corresponding macroblock.
According to another example, if the flag information of an upper level higher than the macroblock level is determined to be "true", the system can obtain an offset value of a current block indicating a difference in average pixel value between the current block and the reference block. In this case, the flag information of the macroblock level or the flag information of the block level may not be employed as necessary. The illumination compensation technique can indicate whether the illumination compensation of each block is performed using the flag information. The illumination compensation technique may also indicate whether the illumination compensation of each block is performed using a specific value such as a motion vector. The above-mentioned example can also be applied to a variety of applications of the illumination compensation technique. In association with the upper scope (i.e., sequence, view, GOP, and picture), the above-mentioned example can indicate whether the illumination compensation of a lower scope is performed using the flag information. The macroblock or block level acting as the lowest scope can effectively indicate whether the illumination compensation is performed using the offset value without using the flag bit. Similar to the method for use of the motion vector, the predictive coding process can be performed. For example, if the predictive coding process is applied to the current block, the offset value of the neighboring block is assigned to an offset value of the current block. If the predictive coding scheme is determined to be the bi-predictive coding scheme, offset values of individual reference blocks are obtained by the calculation of the reference blocks detected from List 0 and List 1. Therefore, in the case of encoding the offset values of the current block, the offset value of each reference is not directly encoded by the offset values of the neighboring blocks, and a residual value is encoded/decoded. The method for predicting the offset value may be determined to be the above-mentioned oftset prediction method or a method for obtaining a median value used for predicting the motion vector. In the case of a direct mode of a bi-directional prediction, supplementary information is not encoded/decoded using the same method as in the motion vector, and the offset values can be obtained by predetermined information.
According to another example, a decoding unit (e.g., H.264-based decoding unit) is used instead of the MVC decoding unit. A view sequence compatible with a conventional decoding unit should be decoded by the conventional decoding unit, such that the "view_IC_flag" information is set to "false" or "0". In this case, there is a need to explain the base-view concept. It should be noted that a single view sequence compatible with the H.264/AVC decoder may be required. Therefore, at least one view, which can be independently decoded, is defined and referred to as a base view. The base view is indicative of a reference view from among several views (i.e., the multiview) . A sequence corresponding to the base view in the MVC scheme is encoded by general video encoding schemes (e.g., MPEG-2, MPEG-4, H.263, and H.264, etc.), such that it is generated in the form of an independent bitstream. The above-mentioned base-view sequence can be compatible with the H.264/AVC scheme, or cannot be compatible with the same. However, the view sequence compatible with the H.264/AVC scheme is always set to the base view.
FIG. 21 is a flow chart illustrating a method for obtaining a motion vector considering an offset value of a current block.
Referring to FIG. 21, the system can obtain an offset value of the current block at step S321. The system searches for a reference block optimally matched with the current block using the offset value at step S322. The system obtains the motion vector from the reference block, and encodes the motion vector at step S323. For the illumination compensation, a variety of factors -are considered during the motion estimation. For example, in the case of a method for comparing a first block with a second block by offsetting average pixel values of the first and second blocks, average pixel values of the two blocks are deducted from pixel values of each block during the motion estimation, such that the similarity between the two blocks can be calculated. In this case, the offset value between the two blocks is independently encoded, such that the costs for the independent encoding are reflected in the motion estimation process. The conventional costs can be calculated by the following equation 23: [Equation 23 ]
COST - SAD + ^MOTION ' GenBH
In the case of using the illumination compensation, the SAD (Sum of Absolute Differences) can be represented by the following equation 24: [Equation 24]
Figure imgf000092_0001
In equation 24, Ic is indicative of a pixel value of
the current block, and I1. is indicative of a pixel value of
the reference block. Mc is indicative of an average pixel
value of the current block, and M1. is indicative of an
average pixel value of the reference block. The offset costs can be included in the above-mentioned SAD calculation process, as denoted by the following equations 25 and 26:
[Equation 25]
COSTIC = SADIC + λM0TI0N ■ GenBit
[ Equation 26 ]
SADIC
Figure imgf000092_0002
With reference to Equations 25 and 26, a is indicative of a weighted coefficient. If the value of a is set to "1", the absolute value of the offset value is reflected. For another method for reflecting the illumination compensation cost, there is a method for reflecting the illumination compensation cost by predicting the number of bits required for encoding the offset value.
The following equation 27 represents a method for predicting the offset coding bit. In this case, the coding bit can be predicted in proportion to the magnitude of an offset residual value.
[Equation 27]
GenBitjr = GenBit + Bitτr
In this case , a new cost can be calculated by the following equation 28 : [Equation 28 ]
Cost = SAD + λMOTION - GenBit IC

Claims

What is Claimed is
1. A method for decoding a multiview video signal, comprising: receiving a bitstream comprising encodings of multiple views of the multiview video signal, each view comprising multiple pictures segmented into multiple segments; extracting flag information associated with a portion of the multiview video signal from the bitstream indicating whether illumination compensation of segments within said portion of the multiview video signal is enabled; and for a portion in which illumination compensation is enabled according to the extracted flag information, extracting from the bitstream a value associated with a segment within the portion and determining from said extracted value whether illumination compensation of the segment is to be performed.
2. The method according to claim 1, wherein the segments comprise image blocks.
3. The method according to claim 2, further comprising, for a first block associated with a value that indicates that illumination compensation is to be performed, obtaining a predictor for performing illumination compensation of the first block using an offset value for illumination compensation of at least one neighboring block adjacent to the first block.
4. The method according to claim 3, wherein an offset value for illumination compensation of a neighboring block is obtained by forming a sum that includes a predictor for illumination compensation of the neighboring block and a residual value.
5. The method according to claim 3, wherein obtaining a predictor for illumination compensation of the first block using an offset value for illumination compensation of at least one neighboring block adjacent to the first block includes selecting the at least one neighboring block according to a predetermined order among the neighboring blocks.
6. The method according to claim 5, wherein selecting the at least one neighboring block according to the predetermined order comprises determining whether one or more conditions are satisfied for a neighboring block in an order in which one or more vertical or horizontal neighbors are followed by one or more diagonal neighbors.
7. The method according to claim 2, wherein the flag information enables illumination compensation for one or more of a sequence, a view, a group of pictures, a picture, and a slice that contains the block.
8. The method according to claim 7, wherein the flag information enables illumination compensation for the slice that contains the block.
9. The method according to claim 7, wherein the extracted value comprises flag information for a macroblock that contains the block or flag information for the block.
10. The method according to claim 9, wherein the extracted value comprises flag information for the macroblock that contains the block.
11. A method for decoding a multiview video signal, comprising: receiving a bitstream comprising encodings of multiple views of the multiview video signal, each view comprising multiple pictures segmented into multiple segments; and obtaining a predictor for illumination compensation of a first segment using an offset value for illumination compensation of at least one neighboring segment adjacent to the first segment, including selecting the at least one neighboring segment according to a predetermined order among the neighboring segments.
12. The method of claim 11, wherein the first segment and the at least one neighboring segment comprise image blocks.
13. The method according to claim 12, wherein an offset value for illumination compensation of a neighboring block is obtained by forming a sum that includes a predictor for illumination compensation of the neighboring block and a residual value.
14. The method according to claim 12, wherein selecting the at least one neighboring block according to the predetermined order comprises determining whether one or more conditions are satisfied for a neighboring block in an order in which one or more vertical or horizontal neighbors are followed by one or more diagonal neighbors.
15. The method according to claim 14, wherein selecting the at least one neighboring block according to the predetermined order comprises determining whether one or more conditions are satisfied for a neighboring block in the order of: a left neighboring block, followed by an upper neighboring block, followed by a right-upper neighboring block, followed by a left-upper neighboring block.
16. The method according to claim 14, wherein determining whether one or more conditions are satisfied for a neighboring block comprises extracting a value associated with the neighboring block from the bitstream indicating whether illumination compensation of the neighboring block is to be performed.
17. The method according to claim 16, wherein the extracted value comprises flag information for a macroblock that contains the block or flag information for the block.
18. The method according to claim 12, wherein obtaining the predictor comprises determining whether to use an offset value for illumination compensation of a single neighboring block or multiple offset values for illumination compensation of respective neighboring blocks.
19. The method according to claim 18, further comprising, when multiple offset values are to be used, obtaining the predictor for performing illumination compensation of the first block by combining the multiple offset values.
20. The method according to claim 19, wherein combining the multiple offset values comprises taking an average or median of the offset values.
21. A method for decoding a multiview video signal, comprising: receiving a bitstream comprising encodings of multiple views of the multiview video signal, each view comprising multiple pictures segmented into multiple segments; obtaining an offset value for illumination compensation of a first segment with respect to a reference picture, wherein the offset value is predicted using an offset value for illumination compensation of at least one neighboring segment determined based on characteristics associated with the neighboring segment; and decoding the bitstream using illumination compensation for the first segment including forming a sum that includes a predictor for pixels of the first segment obtained from the reference picture, a residual for pixels of the first segment, and a corresponding offset value for illumination compensation .
22. The method according to claim 21, wherein the first segment and the at least one neighboring segment comprise image blocks.
23. The method according to claim 22, wherein an offset value for illumination compensation of a neighboring block is obtained by forming a sum that includes a predictor for illumination compensation of the neighboring block and a residual value.
24. The method according to claim 22, further comprising selecting at least one neighboring block based on whether one or more conditions are satisfied for a neighboring block in an order in which one or more vertical or horizontal neighbors are followed by one or more diagonal neighbors.
25. The method according to claim 24, wherein selecting at least one neighboring block comprises determining whether one or more conditions are satisfied for a neighboring block in the order of: a left neighboring block, followed by an upper neighboring block, followed by a right-upper neighboring block, followed by a left-upper neighboring block.
26. The method according to claim 24, wherein determining whether one or more conditions are satisfied for a neighboring block comprises extracting a value associated with the neighboring block from the bitstream indicating whether illumination compensation of the neighboring block is to be performed.
27. The method according to claim 26, wherein the extracted value comprises flag information for a macroblock that contains the block or flag information for the block.
28. The method according to claim 24, wherein selecting at least one neighboring block comprises determining whether to use an offset value for illumination compensation of a single neighboring block or multiple offset values for illumination compensation of respective neighboring blocks.
29. The method according to claim 28, further comprising, when multiple offset values are to be used, obtaining the predictor for performing illumination compensation of the first block by combining the multiple offset values.
30. The method according to claim 29, wherein combining the multiple offset values comprises taking an average or median of the offset values.
31. A method for decoding a multiview video signal, comprising: receiving a bitstream comprising encodings of multiple views of the multiview video signal, each view comprising multiple pictures segmented into multiple segments; obtaining a predictor for illumination compensation of a first segment with respect to a reference picture; determining an offset value for illumination compensation of the first segment including forming a sum that includes the predictor and a residual value; and decoding the bitstream using illumination compensation for the first segment including forming a sum that includes a predictor for pixels of the first segment obtained from the reference picture, a residual for pixels of the first segment, and a corresponding offset value for illumination compensation.
32. The method according to claim 31, wherein the segments comprise image blocks.
33. The method according to claim 32, wherein using illumination compensation for the first segment comprises obtaining an offset value for illumination compensation of a neighboring block by forming a sum that includes a predictor for illumination compensation of the neighboring block and a residual value.
34. The method according to claim 33, further comprising selecting at least one neighboring block based on whether one or more conditions are satisfied for a neighboring block in an order in which one or more vertical or horizontal neighbors are followed by one or more diagonal neighbors.
35. The method according to claim 34, wherein selecting at least one neighboring block comprises determining whether one or more conditions are satisfied for a neighboring block in the order of: a left neighboring block, followed by an upper neighboring block, followed by a right-upper neighboring block, followed by a left-upper neighboring block.
36. The method according to claim 34, wherein determining whether one or more conditions are satisfied for a neighboring block comprises extracting a value associated with the neighboring block from the bitstream indicating whether illumination compensation of the neighboring block is to be performed.
37. The method according to claim 36, wherein the extracted value comprises flag information for a macroblock that contains the block or flag information for the block.
38. The method according to claim 34, wherein selecting at least one neighboring block comprises determining whether to use an offset value for illumination compensation of a single neighboring block or multiple offset values for illumination compensation of respective neighboring blocks.
39. The method according to claim 38, further comprising, when multiple offset values are to be used, obtaining the predictor for performing illumination compensation of the first block by combining the multiple offset values.
40. The method according to claim 39, wherein combining the multiple offset values comprises taking an average or median of the offset values.
41. A method for decoding a multiview video signal, comprising : receiving a bitstream comprising encodings of multiple views of the multiview video signal, each view comprising multiple pictures segmented into multiple segments; and obtaining a predictor for illumination compensation of a first segment with respect to a reference picture using an offset value for illumination compensation of at least one neighboring segment adjacent to the first segment according whether the reference picture associated with the first segment is the same as a reference picture associated with the neighboring segment.
42. The method according to claim 41, wherein the segments comprise image blocks.
43. The method according to claim 42, wherein using illumination compensation for the first segment comprises obtaining an offset value for illumination compensation of a neighboring block by forming a sum that includes a predictor for illumination compensation of the neighboring block and a residual value.
44. The method according to claim 43, further comprising selecting at least one neighboring block based on whether one or more conditions are satisfied for a neighboring block in an order in which one or more vertical or horizontal neighbors are followed by one or more diagonal neighbors.
45. The method according to claim 44, wherein selecting at least one neighboring block comprises determining whether one or more conditions are satisfied for a neighboring block in the order of: a left neighboring block, followed by an upper neighboring block, followed by a right-upper neighboring block, followed by a left-upper neighboring block.
46. The method according to claim 44, wherein determining whether one or more conditions are satisfied for a neighboring block comprises extracting a value associated with the neighboring block from the bitstream indicating whether illumination compensation of the neighboring block is to be performed.
47. The method according to claim 46, wherein the extracted value comprises flag information for a macroblock that contains the block or flag information for the block.
48. The method according to claim 44, wherein selecting at least one neighboring block comprises determining whether to use an offset value for illumination compensation of a single neighboring block or multiple offset values for illumination compensation of respective neighboring blocks.
49. The method according to claim 48, further comprising, when multiple offset values are to be used, obtaining the predictor for performing illumination compensation of the first block by combining the multiple offset values.
50. The method according to claim 49, wherein combining the multiple offset values comprises taking an average or median of the offset values.
PCT/KR2007/000225 2006-01-12 2007-01-12 Processing multiview video WO2007081176A1 (en)

Priority Applications (4)

Application Number Priority Date Filing Date Title
JP2008550241A JP5192393B2 (en) 2006-01-12 2007-01-12 Multi-view video processing
CN200780003083.8A CN101371571B (en) 2006-01-12 2007-01-12 Processing multiview video
EP07700952A EP1982517A4 (en) 2006-01-12 2007-01-12 Processing multiview video
KR1020087019748A KR101276847B1 (en) 2006-01-12 2007-01-12 Processing multiview video

Applications Claiming Priority (24)

Application Number Priority Date Filing Date Title
US75823406P 2006-01-12 2006-01-12
US60/758,234 2006-01-12
KR10-2006-0004956 2006-01-17
KR20060004956 2006-01-17
US75962006P 2006-01-18 2006-01-18
US60/759,620 2006-01-18
US76253406P 2006-01-27 2006-01-27
US60/762,534 2006-01-27
KR10-2006-0027100 2006-03-24
KR20060027100 2006-03-24
US78719306P 2006-03-30 2006-03-30
US60/787,193 2006-03-30
KR1020060037773A KR20070076356A (en) 2006-01-18 2006-04-26 Method and apparatus for coding and decoding of video sequence
KR10-2006-0037773 2006-04-26
US81827406P 2006-07-05 2006-07-05
US60/818,274 2006-07-05
US83008706P 2006-07-12 2006-07-12
US60/830,087 2006-07-12
US83032806P 2006-07-13 2006-07-13
US60/830,328 2006-07-13
KR1020060110338A KR20070076392A (en) 2006-01-18 2006-11-09 A method and apparatus for decoding/encoding a video signal
KR10-2006-0110338 2006-11-09
KR10-2006-0110337 2006-11-09
KR1020060110337A KR20070076391A (en) 2006-01-18 2006-11-09 A method and apparatus for decoding/encoding a video signal

Publications (1)

Publication Number Publication Date
WO2007081176A1 true WO2007081176A1 (en) 2007-07-19

Family

ID=46045583

Family Applications (3)

Application Number Title Priority Date Filing Date
PCT/KR2007/000225 WO2007081176A1 (en) 2006-01-12 2007-01-12 Processing multiview video
PCT/KR2007/000226 WO2007081177A1 (en) 2006-01-12 2007-01-12 Processing multiview video
PCT/KR2007/000228 WO2007081178A1 (en) 2006-01-12 2007-01-12 Processing multiview video

Family Applications After (2)

Application Number Title Priority Date Filing Date
PCT/KR2007/000226 WO2007081177A1 (en) 2006-01-12 2007-01-12 Processing multiview video
PCT/KR2007/000228 WO2007081178A1 (en) 2006-01-12 2007-01-12 Processing multiview video

Country Status (6)

Country Link
US (9) US7831102B2 (en)
EP (3) EP1982517A4 (en)
JP (3) JP5192393B2 (en)
KR (8) KR100953646B1 (en)
DE (1) DE202007019463U1 (en)
WO (3) WO2007081176A1 (en)

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2008005124A2 (en) * 2006-07-05 2008-01-10 Thomson Licensing Methods and apparatus for multi-view video encoding and decoding
JP2008271422A (en) * 2007-04-24 2008-11-06 Ntt Docomo Inc Moving image predictive coding apparatus, method and program, and moving image predictive decoding apparatus, method and program
EP1997318A1 (en) * 2006-03-22 2008-12-03 Electronics and Telecommunications Research Institute Method and apparatus for encoding and decoding the compensated illumination change
WO2010092772A1 (en) * 2009-02-12 2010-08-19 日本電信電話株式会社 Multi-view image encoding method, multi-view image decoding method, multi-view image encoding device, multi-view image decoding device, multi-view image encoding program, and multi-view image decoding program
WO2010095471A1 (en) * 2009-02-23 2010-08-26 日本電信電話株式会社 Multi-view image coding method, multi-view image decoding method, multi-view image coding device, multi-view image decoding device, multi-view image coding program, and multi-view image decoding program
JP2011503979A (en) * 2007-12-13 2011-01-27 メディアテック インコーポレイテッド Improved in-loop fidelity for video compression
JP2011515940A (en) * 2008-03-18 2011-05-19 サムスン エレクトロニクス カンパニー リミテッド Video encoding and decoding method and apparatus
WO2012092777A1 (en) 2011-01-09 2012-07-12 Mediatek Inc. Apparatus and method of sample adaptive offset for video coding
CN104539968A (en) * 2009-08-14 2015-04-22 三星电子株式会社 A method and apparatus for decoding a video
CN104754356A (en) * 2011-11-08 2015-07-01 三星电子株式会社 Method And Apparatus For Motion Vector Determination In Video Encoding Or Decoding
US9432670B2 (en) 2011-01-09 2016-08-30 Mediatek Inc. Apparatus and method of efficient sample adaptive offset
US9485517B2 (en) 2011-04-20 2016-11-01 Qualcomm Incorporated Motion vector prediction with motion vectors from multiple views in multi-view video coding
US10200709B2 (en) 2012-03-16 2019-02-05 Qualcomm Incorporated High-level syntax extensions for high efficiency video coding

Families Citing this family (228)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7068729B2 (en) 2001-12-21 2006-06-27 Digital Fountain, Inc. Multi-stage code generator and decoder for communication systems
US6307487B1 (en) 1998-09-23 2001-10-23 Digital Fountain, Inc. Information additive code generator and decoder for communication systems
US7003035B2 (en) 2002-01-25 2006-02-21 Microsoft Corporation Video coding methods and apparatuses
US20040001546A1 (en) 2002-06-03 2004-01-01 Alexandros Tourapis Spatiotemporal prediction for bidirectionally predictive (B) pictures and motion vector prediction for multi-picture reference motion compensation
US9240810B2 (en) 2002-06-11 2016-01-19 Digital Fountain, Inc. Systems and processes for decoding chain reaction codes through inactivation
US7154952B2 (en) 2002-07-19 2006-12-26 Microsoft Corporation Timestamp-independent motion vector prediction for predictive (P) and bidirectionally predictive (B) pictures
KR101143282B1 (en) 2002-10-05 2012-05-08 디지털 파운튼, 인크. Systematic encoding and decoding of chain reaction codes
US7418651B2 (en) 2004-05-07 2008-08-26 Digital Fountain, Inc. File download and streaming system
US7903737B2 (en) * 2005-11-30 2011-03-08 Mitsubishi Electric Research Laboratories, Inc. Method and system for randomly accessing multiview videos with known prediction dependency
US8401091B2 (en) * 2006-01-09 2013-03-19 Lg Electronics Inc. Inter-layer prediction method for video signal
US20070177671A1 (en) * 2006-01-12 2007-08-02 Lg Electronics Inc. Processing multiview video
KR101276847B1 (en) 2006-01-12 2013-06-18 엘지전자 주식회사 Processing multiview video
JP5192393B2 (en) * 2006-01-12 2013-05-08 エルジー エレクトロニクス インコーポレイティド Multi-view video processing
WO2007095550A2 (en) 2006-02-13 2007-08-23 Digital Fountain, Inc. Streaming and buffering using variable fec overhead and protection periods
US9270414B2 (en) 2006-02-21 2016-02-23 Digital Fountain, Inc. Multiple-field based code generator and decoder for communications systems
US20100091845A1 (en) * 2006-03-30 2010-04-15 Byeong Moon Jeon Method and apparatus for decoding/encoding a video signal
WO2007114611A1 (en) * 2006-03-30 2007-10-11 Lg Electronics Inc. A method and apparatus for decoding/encoding a video signal
US7971129B2 (en) 2006-05-10 2011-06-28 Digital Fountain, Inc. Code generator and decoder for communications systems operating using hybrid codes to allow for multiple efficient users of the communications systems
US9178535B2 (en) 2006-06-09 2015-11-03 Digital Fountain, Inc. Dynamic stream interleaving and sub-stream based delivery
US9432433B2 (en) 2006-06-09 2016-08-30 Qualcomm Incorporated Enhanced block-request streaming system using signaling or block creation
US9209934B2 (en) 2006-06-09 2015-12-08 Qualcomm Incorporated Enhanced block-request streaming using cooperative parallel HTTP and forward error correction
US9419749B2 (en) 2009-08-19 2016-08-16 Qualcomm Incorporated Methods and apparatus employing FEC codes with permanent inactivation of symbols for encoding and decoding processes
US9380096B2 (en) 2006-06-09 2016-06-28 Qualcomm Incorporated Enhanced block-request streaming system for handling low-latency streaming
US9386064B2 (en) 2006-06-09 2016-07-05 Qualcomm Incorporated Enhanced block-request streaming using URL templates and construction rules
WO2007148906A1 (en) * 2006-06-19 2007-12-27 Lg Electronics, Inc. Method and apparatus for processing a vedeo signal
EP2052546A4 (en) * 2006-07-12 2010-03-03 Lg Electronics Inc A method and apparatus for processing a signal
CN101518086B (en) * 2006-07-20 2013-10-30 汤姆森特许公司 Method and apparatus for signaling view scalability in multi-view video coding
US8532178B2 (en) * 2006-08-25 2013-09-10 Lg Electronics Inc. Method and apparatus for decoding/encoding a video signal with inter-view reference picture list construction
EP2090110A2 (en) 2006-10-13 2009-08-19 Thomson Licensing Reference picture list management syntax for multiple view video coding
KR101366092B1 (en) 2006-10-13 2014-02-21 삼성전자주식회사 Method and apparatus for encoding and decoding multi-view image
WO2008048515A2 (en) * 2006-10-13 2008-04-24 Thomson Licensing Method for reference picture management involving multiview video coding
JP5342445B2 (en) * 2006-10-16 2013-11-13 トムソン ライセンシング Method of using network abstraction layer to signal instantaneous decoding refresh during video operation
EP2082584B1 (en) * 2006-10-18 2023-07-19 InterDigital VC Holdings, Inc. Methods, apparatus and storage media for local illumination and color compensation without explicit signaling
US20080095228A1 (en) * 2006-10-20 2008-04-24 Nokia Corporation System and method for providing picture output indications in video coding
WO2008051380A2 (en) * 2006-10-24 2008-05-02 Thomson Licensing Picture identification for multi-view video coding
KR101370287B1 (en) * 2006-11-22 2014-03-07 세종대학교산학협력단 Method and apparatus for deblocking filtering
KR100856411B1 (en) * 2006-12-01 2008-09-04 삼성전자주식회사 Method and apparatus for compensating illumination compensation and method and apparatus for encoding moving picture based on illumination compensation, and method and apparatus for encoding moving picture based on illumination compensation
KR100905723B1 (en) * 2006-12-08 2009-07-01 한국전자통신연구원 System and Method for Digital Real Sense Transmitting/Receiving based on Non-Realtime
KR100922275B1 (en) * 2006-12-15 2009-10-15 경희대학교 산학협력단 Derivation process of a boundary filtering strength and deblocking filtering method and apparatus using the derivation process
BRPI0806237A2 (en) * 2007-01-04 2011-09-06 Thomson Licensing methods and apparatus for multivista information transmitted in high level syntax
JP5467637B2 (en) * 2007-01-04 2014-04-09 トムソン ライセンシング Method and apparatus for reducing coding artifacts for illumination compensation and / or color compensation in multi-view coded video
US8923399B2 (en) * 2007-01-24 2014-12-30 Lg Electronics Inc. Method and an apparatus for processing a video signal
CN101641954B (en) * 2007-03-23 2011-09-14 Lg电子株式会社 A method and an apparatus for decoding/encoding a video signal
US8548261B2 (en) * 2007-04-11 2013-10-01 Samsung Electronics Co., Ltd. Method and apparatus for encoding and decoding multi-view image
BR122018004904B1 (en) 2007-04-12 2019-11-05 Dolby Int Ab video coding and decoding tiling
WO2008140190A1 (en) * 2007-05-14 2008-11-20 Samsung Electronics Co, . Ltd. Method and apparatus for encoding and decoding multi-view image
BRPI0810213A8 (en) * 2007-05-16 2018-12-11 Thomson Licensing methods and apparatus for using slice groups for decoding mvc video encoding information.
KR101244917B1 (en) * 2007-06-11 2013-03-18 삼성전자주식회사 Method and apparatus for compensating illumination compensation and method and apparatus for encoding and decoding video based on illumination compensation
US20080317124A1 (en) * 2007-06-25 2008-12-25 Sukhee Cho Multi-view video coding system, decoding system, bitstream extraction system for decoding base view and supporting view random access
KR101460362B1 (en) * 2007-06-25 2014-11-14 삼성전자주식회사 Method and apparatus for illumination compensation of multi-view video coding
KR20080114482A (en) * 2007-06-26 2008-12-31 삼성전자주식회사 Method and apparatus for illumination compensation of multi-view video coding
BRPI0811469A8 (en) * 2007-06-28 2019-01-22 Thomson Licensing methods and device in an encoder and decoder to support multiview image encoded video single-loop decoding
US8254455B2 (en) 2007-06-30 2012-08-28 Microsoft Corporation Computing collocated macroblock information for direct mode macroblocks
CN101816180B (en) * 2007-08-06 2013-01-16 汤姆森特许公司 Methods and apparatus for motion skip mode with multiple inter-view reference pictures
US20090060043A1 (en) * 2007-08-29 2009-03-05 Geert Nuyttens Multiviewer based on merging of output streams of spatio scalable codecs in a compressed domain
CA2697764A1 (en) 2007-09-12 2009-03-19 Steve Chen Generating and communicating source identification information to enable reliable communications
EP2198620A2 (en) * 2007-10-05 2010-06-23 Thomson Licensing Methods and apparatus for incorporating video usability information (vui) within a multi-view video (mvc) coding system
KR101345287B1 (en) 2007-10-12 2013-12-27 삼성전자주식회사 Scalable video encoding method and apparatus and scalable video decoding method and apparatus
CN101415114B (en) * 2007-10-17 2010-08-25 华为终端有限公司 Method and apparatus for encoding and decoding video, and video encoder and decoder
US8270472B2 (en) * 2007-11-09 2012-09-18 Thomson Licensing Methods and apparatus for adaptive reference filtering (ARF) of bi-predictive pictures in multi-view coded video
KR20090090152A (en) * 2008-02-20 2009-08-25 삼성전자주식회사 Method and apparatus for video encoding and decoding
US20090219985A1 (en) * 2008-02-28 2009-09-03 Vasanth Swaminathan Systems and Methods for Processing Multiple Projections of Video Data in a Single Video File
US8811499B2 (en) * 2008-04-10 2014-08-19 Imagine Communications Corp. Video multiviewer system permitting scrolling of multiple video windows and related methods
KR101591085B1 (en) * 2008-05-19 2016-02-02 삼성전자주식회사 Apparatus and method for generating and playing image file
KR101517768B1 (en) * 2008-07-02 2015-05-06 삼성전자주식회사 Method and apparatus for encoding video and method and apparatus for decoding video
US8326075B2 (en) 2008-09-11 2012-12-04 Google Inc. System and method for video encoding using adaptive loop filter
CN102177718B (en) * 2008-10-07 2014-03-12 艾利森电话股份有限公司 Method and device for generating multi-view media data
KR20100040640A (en) * 2008-10-10 2010-04-20 엘지전자 주식회사 Receiving system and method of processing data
EP2365694B1 (en) * 2008-11-18 2017-01-04 LG Electronics Inc. Method and apparatus for processing image signal
KR101190891B1 (en) * 2008-12-17 2012-10-12 파나소닉 주식회사 Method for forming through electrode, and semiconductor device
CN102257823B (en) * 2008-12-18 2017-03-08 Lg电子株式会社 For showing digital broadcast receiving method and the digital broacast receiver using the method for stereo-picture
MY155999A (en) 2009-01-19 2015-12-31 Panasonic Ip Corp America Coding method,decoding method,coding apparatus,decoding apparatus,program, and integrated circuit
KR101676059B1 (en) 2009-01-26 2016-11-14 톰슨 라이센싱 Frame packing for video coding
WO2010087574A2 (en) 2009-01-28 2010-08-05 Lg Electronics Inc. Broadcast receiver and video data processing method thereof
US8189666B2 (en) * 2009-02-02 2012-05-29 Microsoft Corporation Local picture identifier and computation of co-located information
KR20100089705A (en) * 2009-02-04 2010-08-12 삼성전자주식회사 Apparatus and method for encoding and decoding 3d video
US8270495B2 (en) * 2009-02-13 2012-09-18 Cisco Technology, Inc. Reduced bandwidth off-loading of entropy coding/decoding
US9281847B2 (en) 2009-02-27 2016-03-08 Qualcomm Incorporated Mobile reception of digital video broadcasting—terrestrial services
JP5267886B2 (en) * 2009-04-08 2013-08-21 ソニー株式会社 REPRODUCTION DEVICE, RECORDING MEDIUM, AND INFORMATION PROCESSING METHOD
JP4957823B2 (en) 2009-04-08 2012-06-20 ソニー株式会社 Playback apparatus and playback method
JP4985883B2 (en) * 2009-04-08 2012-07-25 ソニー株式会社 REPRODUCTION DEVICE, REPRODUCTION METHOD, AND RECORDING METHOD
JP4962525B2 (en) * 2009-04-08 2012-06-27 ソニー株式会社 REPRODUCTION DEVICE, REPRODUCTION METHOD, AND PROGRAM
KR101619451B1 (en) * 2009-04-17 2016-05-10 엘지전자 주식회사 Method and apparatus for processing a multiview video signal
RU2550552C2 (en) 2009-04-28 2015-05-10 Панасоник Корпорэйшн Image decoding method and image decoding apparatus
US8411746B2 (en) 2009-06-12 2013-04-02 Qualcomm Incorporated Multiview video coding over MPEG-2 systems
US8780999B2 (en) 2009-06-12 2014-07-15 Qualcomm Incorporated Assembling multiview video coding sub-BITSTREAMS in MPEG-2 systems
KR101631270B1 (en) * 2009-06-19 2016-06-16 삼성전자주식회사 Method and apparatus for filtering image by using pseudo-random filter
KR20110007928A (en) * 2009-07-17 2011-01-25 삼성전자주식회사 Method and apparatus for encoding/decoding multi-view picture
US8948241B2 (en) * 2009-08-07 2015-02-03 Qualcomm Incorporated Signaling characteristics of an MVC operation point
US9288010B2 (en) 2009-08-19 2016-03-15 Qualcomm Incorporated Universal file delivery methods for providing unequal error protection and bundled file delivery services
EP2302933A1 (en) * 2009-09-17 2011-03-30 Mitsubishi Electric R&D Centre Europe B.V. Weighted motion compensation of video
US9917874B2 (en) 2009-09-22 2018-03-13 Qualcomm Incorporated Enhanced block-request streaming using block partitioning or request controls for improved client-side handling
WO2011068807A1 (en) * 2009-12-01 2011-06-09 Divx, Llc System and method for determining bit stream compatibility
KR20110068792A (en) 2009-12-16 2011-06-22 한국전자통신연구원 Adaptive image coding apparatus and method
JP5722349B2 (en) 2010-01-29 2015-05-20 トムソン ライセンシングＴｈｏｍｓｏｎ Ｌｉｃｅｎｓｉｎｇ Block-based interleaving
WO2011105337A1 (en) * 2010-02-24 2011-09-01 日本電信電話株式会社 Multiview video coding method, multiview video decoding method, multiview video coding device, multiview video decoding device, and program
KR101289269B1 (en) * 2010-03-23 2013-07-24 한국전자통신연구원 An apparatus and method for displaying image data in image system
JP2011216965A (en) * 2010-03-31 2011-10-27 Sony Corp Information processing apparatus, information processing method, reproduction apparatus, reproduction method, and program
US20110280311A1 (en) 2010-05-13 2011-11-17 Qualcomm Incorporated One-stream coding for asymmetric stereo video
US9510009B2 (en) * 2010-05-20 2016-11-29 Thomson Licensing Methods and apparatus for adaptive motion vector candidate ordering for video encoding and decoding
JP5387520B2 (en) * 2010-06-25 2014-01-15 ソニー株式会社 Information processing apparatus and information processing method
US9485546B2 (en) 2010-06-29 2016-11-01 Qualcomm Incorporated Signaling video samples for trick mode video representations
JP5392199B2 (en) * 2010-07-09 2014-01-22 ソニー株式会社 Image processing apparatus and method
US9185439B2 (en) 2010-07-15 2015-11-10 Qualcomm Incorporated Signaling data for multiplexing video components
US9596447B2 (en) * 2010-07-21 2017-03-14 Qualcomm Incorporated Providing frame packing type information for video coding
US9456015B2 (en) 2010-08-10 2016-09-27 Qualcomm Incorporated Representation groups for network streaming of coded multimedia data
CN103155558B (en) * 2010-08-11 2017-05-10 Ge视频压缩有限责任公司 Multi-view signal codec
WO2012050832A1 (en) 2010-09-28 2012-04-19 Google Inc. Systems and methods utilizing efficient video compression techniques for providing static image data
US9532059B2 (en) 2010-10-05 2016-12-27 Google Technology Holdings LLC Method and apparatus for spatial scalability for video coding
CA2810899C (en) 2010-10-05 2016-08-09 General Instrument Corporation Coding and decoding utilizing adaptive context model selection with zigzag scan
US20130250056A1 (en) * 2010-10-06 2013-09-26 Nomad3D Sas Multiview 3d compression format and algorithms
EP2630799A4 (en) * 2010-10-20 2014-07-02 Nokia Corp Method and device for video coding and decoding
CN107105270B (en) 2010-12-13 2020-04-24 韩国电子通信研究院 Inter-frame prediction method
GB2486692B (en) * 2010-12-22 2014-04-16 Canon Kk Method for encoding a video sequence and associated encoding device
US20120189060A1 (en) * 2011-01-20 2012-07-26 Industry-Academic Cooperation Foundation, Yonsei University Apparatus and method for encoding and decoding motion information and disparity information
US9215473B2 (en) * 2011-01-26 2015-12-15 Qualcomm Incorporated Sub-slices in video coding
US9270299B2 (en) 2011-02-11 2016-02-23 Qualcomm Incorporated Encoding and decoding using elastic codes with flexible source block mapping
KR20120095610A (en) * 2011-02-21 2012-08-29 삼성전자주식회사 Method and apparatus for encoding and decoding multi-view video
KR20120095611A (en) * 2011-02-21 2012-08-29 삼성전자주식회사 Method and apparatus for encoding/decoding multi view video
US8938001B1 (en) 2011-04-05 2015-01-20 Google Inc. Apparatus and method for coding using combinations
US8781004B1 (en) 2011-04-07 2014-07-15 Google Inc. System and method for encoding video using variable loop filter
US8780971B1 (en) 2011-04-07 2014-07-15 Google, Inc. System and method of encoding using selectable loop filters
US8780996B2 (en) 2011-04-07 2014-07-15 Google, Inc. System and method for encoding and decoding video data
US8989256B2 (en) 2011-05-25 2015-03-24 Google Inc. Method and apparatus for using segmentation-based coding of prediction information
SI3481066T1 (en) * 2011-06-28 2021-10-29 Lg Electronics Inc Method for deriving a motion vector predictor
US8879826B2 (en) * 2011-07-05 2014-11-04 Texas Instruments Incorporated Method, system and computer program product for switching between 2D and 3D coding of a video sequence of images
US9521418B2 (en) * 2011-07-22 2016-12-13 Qualcomm Incorporated Slice header three-dimensional video extension for slice header prediction
US11496760B2 (en) 2011-07-22 2022-11-08 Qualcomm Incorporated Slice header prediction for depth maps in three-dimensional video codecs
US8891616B1 (en) 2011-07-27 2014-11-18 Google Inc. Method and apparatus for entropy encoding based on encoding cost
US9674525B2 (en) 2011-07-28 2017-06-06 Qualcomm Incorporated Multiview video coding
US9635355B2 (en) 2011-07-28 2017-04-25 Qualcomm Incorporated Multiview video coding
US9288505B2 (en) * 2011-08-11 2016-03-15 Qualcomm Incorporated Three-dimensional video with asymmetric spatial resolution
CN108989847B (en) 2011-08-30 2021-03-09 帝威视有限公司 System and method for encoding and streaming video
US8818171B2 (en) 2011-08-30 2014-08-26 Kourosh Soroushian Systems and methods for encoding alternative streams of video for playback on playback devices having predetermined display aspect ratios and network connection maximum data rates
US9253233B2 (en) 2011-08-31 2016-02-02 Qualcomm Incorporated Switch signaling methods providing improved switching between representations for adaptive HTTP streaming
US8885706B2 (en) 2011-09-16 2014-11-11 Google Inc. Apparatus and methodology for a video codec system with noise reduction capability
US9420307B2 (en) 2011-09-23 2016-08-16 Qualcomm Incorporated Coding reference pictures for a reference picture set
US9843844B2 (en) 2011-10-05 2017-12-12 Qualcomm Incorporated Network streaming of media data
US9338463B2 (en) 2011-10-06 2016-05-10 Synopsys, Inc. Visual quality measure for real-time video processing
US9781449B2 (en) * 2011-10-06 2017-10-03 Synopsys, Inc. Rate distortion optimization in image and video encoding
US9712819B2 (en) 2011-10-12 2017-07-18 Lg Electronics Inc. Image encoding method and image decoding method
US8787688B2 (en) * 2011-10-13 2014-07-22 Sharp Laboratories Of America, Inc. Tracking a reference picture based on a designated picture on an electronic device
US8768079B2 (en) 2011-10-13 2014-07-01 Sharp Laboratories Of America, Inc. Tracking a reference picture on an electronic device
US8855433B2 (en) * 2011-10-13 2014-10-07 Sharp Kabushiki Kaisha Tracking a reference picture based on a designated picture on an electronic device
US9124895B2 (en) 2011-11-04 2015-09-01 Qualcomm Incorporated Video coding with network abstraction layer units that include multiple encoded picture partitions
US9077998B2 (en) 2011-11-04 2015-07-07 Qualcomm Incorporated Padding of segments in coded slice NAL units
WO2013067942A1 (en) * 2011-11-08 2013-05-16 华为技术有限公司 Intra-frame prediction method and device
US9485503B2 (en) 2011-11-18 2016-11-01 Qualcomm Incorporated Inside view motion prediction among texture and depth view components
US9247257B1 (en) 2011-11-30 2016-01-26 Google Inc. Segmentation based entropy encoding and decoding
US9258559B2 (en) 2011-12-20 2016-02-09 Qualcomm Incorporated Reference picture list construction for multi-view and three-dimensional video coding
JP6394966B2 (en) 2012-01-20 2018-09-26 サン パテント トラスト Encoding method, decoding method, encoding device, and decoding device using temporal motion vector prediction
MX2014008982A (en) * 2012-01-31 2014-08-27 Sony Corp Image processing apparatus and image processing method.
EP3829177A1 (en) 2012-02-03 2021-06-02 Sun Patent Trust Image coding method, image decoding method, image coding apparatus, image decoding apparatus, and image coding and decoding apparatus
US9094681B1 (en) 2012-02-28 2015-07-28 Google Inc. Adaptive segmentation
US9131073B1 (en) 2012-03-02 2015-09-08 Google Inc. Motion estimation aided noise reduction
WO2013128832A1 (en) 2012-03-02 2013-09-06 パナソニック株式会社 Image coding method, image decoding method, image coding device, image decoding device, and image coding-decoding device
GB2500023A (en) * 2012-03-06 2013-09-11 Queen Mary & Westfield College Coding and Decoding a Video Signal Including Generating and Using a Modified Residual and/or Modified Prediction Signal
JP6421931B2 (en) 2012-03-06 2018-11-14 サン パテント トラスト Moving picture coding method and moving picture coding apparatus
US11039138B1 (en) 2012-03-08 2021-06-15 Google Llc Adaptive coding of prediction modes using probability distributions
US20130243085A1 (en) * 2012-03-15 2013-09-19 Samsung Electronics Co., Ltd. Method of multi-view video coding and decoding based on local illumination and contrast compensation of reference frames without extra bitrate overhead
KR102115323B1 (en) * 2012-03-16 2020-05-26 엘지전자 주식회사 Method for storing image information, method for parsing image information and apparatus using same
US9503720B2 (en) 2012-03-16 2016-11-22 Qualcomm Incorporated Motion vector coding and bi-prediction in HEVC and its extensions
US9294226B2 (en) 2012-03-26 2016-03-22 Qualcomm Incorporated Universal object delivery and template-based file delivery
JP2013247651A (en) * 2012-05-29 2013-12-09 Canon Inc Coding apparatus, coding method, and program
JP6000670B2 (en) 2012-06-11 2016-10-05 キヤノン株式会社 Image processing apparatus and image processing method
US9781447B1 (en) 2012-06-21 2017-10-03 Google Inc. Correlation based inter-plane prediction encoding and decoding
US20140003799A1 (en) * 2012-06-30 2014-01-02 Divx, Llc Systems and methods for decoding a video sequence encoded using predictions that include references to frames in reference segments from different video sequences
US10452715B2 (en) 2012-06-30 2019-10-22 Divx, Llc Systems and methods for compressing geotagged video
RU2608354C2 (en) * 2012-07-02 2017-01-18 Самсунг Электроникс Ко., Лтд. Method and apparatus for encoding video and method and apparatus for decoding video determining inter-prediction reference picture list depending on block size
US9774856B1 (en) 2012-07-02 2017-09-26 Google Inc. Adaptive stochastic entropy coding
RU2510944C2 (en) * 2012-07-03 2014-04-10 Корпорация "САМСУНГ ЭЛЕКТРОНИКС Ко., Лтд." Method of encoding/decoding multi-view video sequence based on adaptive local adjustment of brightness of key frames without transmitting additional parameters (versions)
JP5885604B2 (en) 2012-07-06 2016-03-15 株式会社Ｎｔｔドコモ Moving picture predictive coding apparatus, moving picture predictive coding method, moving picture predictive coding program, moving picture predictive decoding apparatus, moving picture predictive decoding method, and moving picture predictive decoding program
IN2014KN03053A (en) * 2012-07-11 2015-05-08 Lg Electronics Inc
US9344729B1 (en) 2012-07-11 2016-05-17 Google Inc. Selective prediction signal filtering
KR101638752B1 (en) * 2012-07-27 2016-07-11 미디어텍 인크. Method of constrain disparity vector derivation in 3d video coding
US9332276B1 (en) 2012-08-09 2016-05-03 Google Inc. Variable-sized super block based direct prediction mode
US9167268B1 (en) 2012-08-09 2015-10-20 Google Inc. Second-order orthogonal spatial intra prediction
US9380298B1 (en) 2012-08-10 2016-06-28 Google Inc. Object-based intra-prediction
US9344742B2 (en) 2012-08-10 2016-05-17 Google Inc. Transform-domain intra prediction
EP2888881A4 (en) 2012-08-23 2016-08-31 Mediatek Inc Method and apparatus of interlayer texture prediction
US20140079116A1 (en) * 2012-09-20 2014-03-20 Qualcomm Incorporated Indication of interlaced video data for video coding
US9554146B2 (en) 2012-09-21 2017-01-24 Qualcomm Incorporated Indication and activation of parameter sets for video coding
CN108429917B (en) 2012-09-29 2022-04-29 华为技术有限公司 Video encoding and decoding methods, devices and systems
US9369732B2 (en) 2012-10-08 2016-06-14 Google Inc. Lossless intra-prediction video coding
WO2014058210A1 (en) * 2012-10-08 2014-04-17 삼성전자 주식회사 Method and apparatus for coding video stream according to inter-layer prediction of multi-view video, and method and apparatus for decoding video stream according to inter-layer prediction of multi-view video
TW201415898A (en) * 2012-10-09 2014-04-16 Sony Corp Image-processing device and method
US9774927B2 (en) * 2012-12-21 2017-09-26 Telefonaktiebolaget L M Ericsson (Publ) Multi-layer video stream decoding
WO2014103606A1 (en) * 2012-12-26 2014-07-03 シャープ株式会社 Image decoding device
US9628790B1 (en) 2013-01-03 2017-04-18 Google Inc. Adaptive composite intra prediction for image and video compression
US9509998B1 (en) 2013-04-04 2016-11-29 Google Inc. Conditional predictive multi-symbol run-length coding
CN104104958B (en) * 2013-04-08 2017-08-25 联发科技（新加坡）私人有限公司 Picture decoding method and its picture decoding apparatus
US9930363B2 (en) * 2013-04-12 2018-03-27 Nokia Technologies Oy Harmonized inter-view and view synthesis prediction for 3D video coding
KR102105323B1 (en) * 2013-04-15 2020-04-28 인텔렉추얼디스커버리 주식회사 A method for adaptive illuminance compensation based on object and an apparatus using it
JP6360053B2 (en) * 2013-06-18 2018-07-18 シャープ株式会社 Illuminance compensation device, image decoding device, image coding device
US10284858B2 (en) * 2013-10-15 2019-05-07 Qualcomm Incorporated Support of multi-mode extraction for multi-layer video codecs
US9392288B2 (en) 2013-10-17 2016-07-12 Google Inc. Video coding using scatter-based scan tables
US9179151B2 (en) 2013-10-18 2015-11-03 Google Inc. Spatial proximity context entropy coding
FR3014278A1 (en) * 2013-11-29 2015-06-05 Orange IMAGE ENCODING AND DECODING METHOD, IMAGE ENCODING AND DECODING DEVICE AND CORRESPONDING COMPUTER PROGRAMS
US10554967B2 (en) 2014-03-21 2020-02-04 Futurewei Technologies, Inc. Illumination compensation (IC) refinement based on positional pairings among pixels
US10102613B2 (en) 2014-09-25 2018-10-16 Google Llc Frequency-domain denoising
WO2016070363A1 (en) * 2014-11-05 2016-05-12 Mediatek Singapore Pte. Ltd. Merge with inter prediction offset
US9871967B2 (en) * 2015-01-22 2018-01-16 Huddly As Video transmission based on independently encoded background updates
US10887597B2 (en) * 2015-06-09 2021-01-05 Qualcomm Incorporated Systems and methods of determining illumination compensation parameters for video coding
US10356416B2 (en) 2015-06-09 2019-07-16 Qualcomm Incorporated Systems and methods of determining illumination compensation status for video coding
PL412844A1 (en) 2015-06-25 2017-01-02 Politechnika Poznańska System and method of coding of the exposed area in the multi-video sequence data stream
US10375413B2 (en) * 2015-09-28 2019-08-06 Qualcomm Incorporated Bi-directional optical flow for video coding
US10148989B2 (en) 2016-06-15 2018-12-04 Divx, Llc Systems and methods for encoding video content
EP3503553A4 (en) * 2016-09-22 2020-07-29 LG Electronics Inc. -1- Illumination compensation-based inter-prediction method and apparatus in image coding system
KR102147447B1 (en) * 2016-09-22 2020-08-24 엘지전자 주식회사 Inter prediction method and apparatus in video coding system
US10742979B2 (en) * 2016-12-21 2020-08-11 Arris Enterprises Llc Nonlinear local activity for adaptive quantization
KR20180074000A (en) * 2016-12-23 2018-07-03 삼성전자주식회사 Method of decoding video data, video decoder performing the same, method of encoding video data, and video encoder performing the same
EP3468194A1 (en) * 2017-10-05 2019-04-10 Thomson Licensing Decoupled mode inference and prediction
JP7248664B2 (en) 2017-10-05 2023-03-29 インターデジタル ヴイシー ホールディングス， インコーポレイテッド Method and apparatus for adaptive illumination compensation in video encoding and decoding
EP3468198A1 (en) * 2017-10-05 2019-04-10 Thomson Licensing Method and apparatus for video encoding and decoding based on illumination compensation
US10652571B2 (en) 2018-01-25 2020-05-12 Qualcomm Incorporated Advanced motion vector prediction speedups for video coding
US10958928B2 (en) * 2018-04-10 2021-03-23 Qualcomm Incorporated Decoder-side motion vector derivation for video coding
CN112653895B (en) * 2018-05-16 2022-08-02 华为技术有限公司 Motion vector acquisition method, device, equipment and computer readable storage medium
BR112020026743A2 (en) * 2018-07-06 2021-03-30 Mitsubishi Electric Corporation METHOD, DECODER, AND SYSTEM
US11140418B2 (en) * 2018-07-17 2021-10-05 Qualcomm Incorporated Block-based adaptive loop filter design and signaling
CN111263147B (en) 2018-12-03 2023-02-14 华为技术有限公司 Inter-frame prediction method and related device
CN111726598B (en) * 2019-03-19 2022-09-16 浙江大学 Image processing method and device
CN110139112B (en) * 2019-04-29 2022-04-05 暨南大学 Video coding method based on JND model
KR20210066282A (en) 2019-11-28 2021-06-07 삼성전자주식회사 Display apparatus and control method for the same
WO2021108913A1 (en) * 2019-12-04 2021-06-10 Studio Thinkwell Montréal Inc. Video system, method for calibrating the video system and method for capturing an image using the video system
KR102475334B1 (en) * 2020-01-13 2022-12-07 한국전자통신연구원 Video encoding/decoding method and apparatus
US11375231B2 (en) * 2020-01-14 2022-06-28 Tencent America LLC Method and apparatus for video coding
US11412256B2 (en) 2020-04-08 2022-08-09 Tencent America LLC Method and apparatus for video coding
US20230024288A1 (en) * 2021-07-13 2023-01-26 Tencent America LLC Feature-based multi-view representation and coding

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030202592A1 (en) * 2002-04-20 2003-10-30 Sohn Kwang Hoon Apparatus for encoding a multi-view moving picture
EP1515550A1 (en) * 2002-06-20 2005-03-16 Sony Corporation Decoding apparatus and decoding method

Family Cites Families (42)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0698312A (en) * 1992-09-16 1994-04-08 Fujitsu Ltd High efficiency picture coding system
ZA951617B (en) 1994-03-04 1997-02-27 Lilly Co Eli Antithrombotic agents.
US6055012A (en) 1995-12-29 2000-04-25 Lucent Technologies Inc. Digital multi-view video compression with complexity and compatibility constraints
CN1134172C (en) 1997-01-13 2004-01-07 皇家菲利浦电子有限公司 Emboding supplemental data in digital video signal
JPH11252552A (en) 1998-03-05 1999-09-17 Sony Corp Compression coding method and compression coder for video signal, and multiplexing method and multiplexer for compression coded data
US6167084A (en) 1998-08-27 2000-12-26 Motorola, Inc. Dynamic bit allocation for statistical multiplexing of compressed and uncompressed digital video signals
EP1280348A4 (en) * 2000-04-21 2004-10-06 Sony Corp Information processing apparatus and method, program, and recorded medium
KR100375708B1 (en) * 2000-10-28 2003-03-15 전자부품연구원 3D Stereosc opic Multiview Video System and Manufacturing Method
KR100397511B1 (en) * 2001-11-21 2003-09-13 한국전자통신연구원 The processing system and it's method for the stereoscopic/multiview Video
EP1418762A1 (en) 2002-05-22 2004-05-12 Matsushita Electric Industrial Co., Ltd. Moving image encoding method, moving image decoding method, and data recording medium
PT1549159E (en) 2002-06-12 2008-12-23 Coca Cola Co Beverages containing plant sterols
KR100997166B1 (en) 2002-06-20 2010-11-29 소니 주식회사 Decoding device and decoding method
KR20040001354A (en) 2002-06-27 2004-01-07 주식회사 케이티 Method for Wireless LAN Service in Wide Area
KR100475060B1 (en) 2002-08-07 2005-03-10 한국전자통신연구원 The multiplexing method and its device according to user's request for multi-view 3D video
KR100751422B1 (en) * 2002-12-27 2007-08-23 한국전자통신연구원 A Method of Coding and Decoding Stereoscopic Video and A Apparatus for Coding and Decoding the Same
US7489342B2 (en) 2004-12-17 2009-02-10 Mitsubishi Electric Research Laboratories, Inc. Method and system for managing reference pictures in multiview videos
US7286689B2 (en) 2003-06-07 2007-10-23 Hewlett-Packard Development Company, L.P. Motion estimation for compression of calibrated multi-view image sequences
EP1642236A1 (en) 2003-06-30 2006-04-05 Koninklijke Philips Electronics N.V. System and method for video processing using overcomplete wavelet coding and circular prediction mapping
US7778328B2 (en) 2003-08-07 2010-08-17 Sony Corporation Semantics-based motion estimation for multi-view video coding
CN1212014C (en) 2003-08-18 2005-07-20 北京工业大学 Video coding method based on time-space domain correlation quick movement estimate
US7613344B2 (en) 2003-12-08 2009-11-03 Electronics And Telecommunications Research Institute System and method for encoding and decoding an image using bitstream map and recording medium thereof
KR100987775B1 (en) 2004-01-20 2010-10-13 삼성전자주식회사 3 Dimensional coding method of video
KR100679740B1 (en) * 2004-06-25 2007-02-07 학교법인연세대학교 Method for Coding/Decoding for Multiview Sequence where View Selection is Possible
US7444664B2 (en) * 2004-07-27 2008-10-28 Microsoft Corp. Multi-view video format
US7671893B2 (en) * 2004-07-27 2010-03-02 Microsoft Corp. System and method for interactive multi-view video
KR100584603B1 (en) 2004-08-03 2006-05-30 학교법인 대양학원 Direct mode motion prediction method and apparatus for multi-view video
US7924923B2 (en) 2004-11-30 2011-04-12 Humax Co., Ltd. Motion estimation and compensation method and device adaptive to change in illumination
EP2538674A1 (en) 2004-12-10 2012-12-26 Electronics and Telecommunications Research Institute Apparatus for universal coding for multi-view video
US7468745B2 (en) * 2004-12-17 2008-12-23 Mitsubishi Electric Research Laboratories, Inc. Multiview video decomposition and encoding
US7710462B2 (en) * 2004-12-17 2010-05-04 Mitsubishi Electric Research Laboratories, Inc. Method for randomly accessing multiview videos
US7728878B2 (en) * 2004-12-17 2010-06-01 Mitsubishi Electric Research Labortories, Inc. Method and system for processing multiview videos for view synthesis using side information
US8644386B2 (en) 2005-09-22 2014-02-04 Samsung Electronics Co., Ltd. Method of estimating disparity vector, and method and apparatus for encoding and decoding multi-view moving picture using the disparity vector estimation method
KR101276720B1 (en) * 2005-09-29 2013-06-19 삼성전자주식회사 Method for predicting disparity vector using camera parameter, apparatus for encoding and decoding muti-view image using method thereof, and a recording medium having a program to implement thereof
CN101292538B (en) * 2005-10-19 2012-11-28 汤姆森特许公司 Multi-view video coding using scalable video coding
ZA200805337B (en) * 2006-01-09 2009-11-25 Thomson Licensing Method and apparatus for providing reduced resolution update mode for multiview video coding
CN101371571B (en) 2006-01-12 2013-06-19 Lg电子株式会社 Processing multiview video
JP5192393B2 (en) * 2006-01-12 2013-05-08 エルジー エレクトロニクス インコーポレイティド Multi-view video processing
WO2007114611A1 (en) * 2006-03-30 2007-10-11 Lg Electronics Inc. A method and apparatus for decoding/encoding a video signal
EP2082584B1 (en) * 2006-10-18 2023-07-19 InterDigital VC Holdings, Inc. Methods, apparatus and storage media for local illumination and color compensation without explicit signaling
BRPI0811469A8 (en) * 2007-06-28 2019-01-22 Thomson Licensing methods and device in an encoder and decoder to support multiview image encoded video single-loop decoding
US8665958B2 (en) * 2008-01-29 2014-03-04 Electronics And Telecommunications Research Institute Method and apparatus for encoding and decoding video signal using motion compensation based on affine transformation
US8130277B2 (en) * 2008-02-20 2012-03-06 Aricent Group Method and system for intelligent and efficient camera motion estimation for video stabilization

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030202592A1 (en) * 2002-04-20 2003-10-30 Sohn Kwang Hoon Apparatus for encoding a multi-view moving picture
EP1515550A1 (en) * 2002-06-20 2005-03-16 Sony Corporation Decoding apparatus and decoding method

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
JOAQUIN L'OPEZ; JAE HOON KIM; ANTONIO ORTEGA; GEORGE CHEN: "Block-based Illumination Compensation and Search Techniques for Multiview Video Coding", PICTURE CODING SYMPOSIUM 2004, 17 December 2004 (2004-12-17)
See also references of EP1982517A4

Cited By (31)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1997318A1 (en) * 2006-03-22 2008-12-03 Electronics and Telecommunications Research Institute Method and apparatus for encoding and decoding the compensated illumination change
EP1997318A4 (en) * 2006-03-22 2011-04-06 Korea Electronics Telecomm Method and apparatus for encoding and decoding the compensated illumination change
WO2008005124A2 (en) * 2006-07-05 2008-01-10 Thomson Licensing Methods and apparatus for multi-view video encoding and decoding
WO2008005124A3 (en) * 2006-07-05 2008-04-24 Thomson Licensing Methods and apparatus for multi-view video encoding and decoding
JP2008271422A (en) * 2007-04-24 2008-11-06 Ntt Docomo Inc Moving image predictive coding apparatus, method and program, and moving image predictive decoding apparatus, method and program
JP2011503979A (en) * 2007-12-13 2011-01-27 メディアテック インコーポレイテッド Improved in-loop fidelity for video compression
US10327010B2 (en) 2007-12-13 2019-06-18 Hfi Innovation Inc. In-loop fidelity enhancement for video compression
JP2011515940A (en) * 2008-03-18 2011-05-19 サムスン エレクトロニクス カンパニー リミテッド Video encoding and decoding method and apparatus
WO2010092772A1 (en) * 2009-02-12 2010-08-19 日本電信電話株式会社 Multi-view image encoding method, multi-view image decoding method, multi-view image encoding device, multi-view image decoding device, multi-view image encoding program, and multi-view image decoding program
CN102308584A (en) * 2009-02-12 2012-01-04 日本电信电话株式会社 Multi-view image encoding method, multi-view image decoding method, multi-view image encoding device, multi-view image decoding device, multi-view image encoding program, and multi-view image decoding program
RU2525850C2 (en) * 2009-02-12 2014-08-20 Ниппон Телеграф Энд Телефон Корпорейшн Multiview image encoding method, multiview image decoding method, multiview image encoder, multiview image decoder, computer-readable medium having multiview image encoding programme and computer-readable medium having multiview image decoding programme
KR101287458B1 (en) 2009-02-12 2013-07-19 니폰덴신뎅와 가부시키가이샤 Multi-view image encoding method, multi-view image decoding method, multi-view image encoding device, multi-view image decoding device, multi-view image encoding program, and multi-view image decoding program
US8532411B2 (en) 2009-02-12 2013-09-10 Nippon Telegraph And Telephone Corporation Multi-view image coding method, multi-view image decoding method, multi-view image coding device, multi-view image decoding device, multi-view image coding program, and multi-view image decoding program
KR101344425B1 (en) 2009-02-23 2013-12-23 고쿠리츠 다이가쿠 호우징 나고야 다이가쿠 Multi-view image coding method, multi-view image decoding method, multi-view image coding device, multi-view image decoding device, multi-view image coding program, and multi-view image decoding program
US8548228B2 (en) 2009-02-23 2013-10-01 Nippon Telegraph And Telephone Corporation Multi-view image coding method, multi-view image decoding method, multi-view image coding device, multi-view image decoding device, multi-view image coding program, and multi-view image decoding program
WO2010095471A1 (en) * 2009-02-23 2010-08-26 日本電信電話株式会社 Multi-view image coding method, multi-view image decoding method, multi-view image coding device, multi-view image decoding device, multi-view image coding program, and multi-view image decoding program
CN104539968A (en) * 2009-08-14 2015-04-22 三星电子株式会社 A method and apparatus for decoding a video
USRE48224E1 (en) 2009-08-14 2020-09-22 Samsung Electronics Co., Ltd. Method and apparatus for encoding video in consideration of scanning order of coding units having hierarchical structure, and method and apparatus for decoding video in consideration of scanning order of coding units having hierarchical structure
CN104539968B (en) * 2009-08-14 2017-12-22 三星电子株式会社 Method and apparatus for being decoded to video
EP3242483A1 (en) * 2011-01-09 2017-11-08 HFI Innovation Inc. Apparatus and method of sample adaptive offset for video coding
WO2012092777A1 (en) 2011-01-09 2012-07-12 Mediatek Inc. Apparatus and method of sample adaptive offset for video coding
EP2661891A4 (en) * 2011-01-09 2016-07-06 Mediatek Inc Apparatus and method of sample adaptive offset for video coding
US9432670B2 (en) 2011-01-09 2016-08-30 Mediatek Inc. Apparatus and method of efficient sample adaptive offset
US9641863B2 (en) 2011-01-09 2017-05-02 Hfi Innovation Inc. Apparatus and method of sample adaptive offset for video coding
US9485517B2 (en) 2011-04-20 2016-11-01 Qualcomm Incorporated Motion vector prediction with motion vectors from multiple views in multi-view video coding
CN104811727A (en) * 2011-11-08 2015-07-29 三星电子株式会社 Method and device for determining motion vector for video coding or video decoding
US9451282B2 (en) 2011-11-08 2016-09-20 Samsung Electronics Co., Ltd. Method and apparatus for motion vector determination in video encoding or decoding
CN104811727B (en) * 2011-11-08 2018-02-16 三星电子株式会社 The method and apparatus determined for the motion vector in Video coding or decoding
US9332273B2 (en) 2011-11-08 2016-05-03 Samsung Electronics Co., Ltd. Method and apparatus for motion vector determination in video encoding or decoding
CN104754356A (en) * 2011-11-08 2015-07-01 三星电子株式会社 Method And Apparatus For Motion Vector Determination In Video Encoding Or Decoding
US10200709B2 (en) 2012-03-16 2019-02-05 Qualcomm Incorporated High-level syntax extensions for high efficiency video coding

Also Published As

Publication number Publication date
JP2009536793A (en) 2009-10-15
WO2007081177A1 (en) 2007-07-19
JP5199123B2 (en) 2013-05-15
KR20090099590A (en) 2009-09-22
KR20090099588A (en) 2009-09-22
JP5192393B2 (en) 2013-05-08
KR100943913B1 (en) 2010-03-03
DE202007019463U1 (en) 2012-10-09
US20070177674A1 (en) 2007-08-02
US20070177810A1 (en) 2007-08-02
US20070177672A1 (en) 2007-08-02
EP1982517A1 (en) 2008-10-22
EP1982518A1 (en) 2008-10-22
KR100943912B1 (en) 2010-03-03
WO2007081178A1 (en) 2007-07-19
JP2009523356A (en) 2009-06-18
US7970221B2 (en) 2011-06-28
US7856148B2 (en) 2010-12-21
JP2009523355A (en) 2009-06-18
KR20090099589A (en) 2009-09-22
US7831102B2 (en) 2010-11-09
KR100934677B1 (en) 2009-12-31
US20070177811A1 (en) 2007-08-02
KR100943914B1 (en) 2010-03-03
KR100934676B1 (en) 2009-12-31
US8115804B2 (en) 2012-02-14
KR20090099097A (en) 2009-09-21
US20070177673A1 (en) 2007-08-02
US20120121015A1 (en) 2012-05-17
KR100947234B1 (en) 2010-03-12
US20070177812A1 (en) 2007-08-02
KR100943915B1 (en) 2010-03-03
EP1977593A1 (en) 2008-10-08
KR20080094047A (en) 2008-10-22
KR20090099098A (en) 2009-09-21
KR100953646B1 (en) 2010-04-21
US8553073B2 (en) 2013-10-08
KR20080094046A (en) 2008-10-22
EP1982517A4 (en) 2010-06-16
US20090310676A1 (en) 2009-12-17
EP1982518A4 (en) 2010-06-16
KR20090099591A (en) 2009-09-22
JP5199124B2 (en) 2013-05-15
US7817866B2 (en) 2010-10-19
US20070177813A1 (en) 2007-08-02
EP1977593A4 (en) 2010-06-16
US7817865B2 (en) 2010-10-19
US8154585B2 (en) 2012-04-10
DE202007019463U8 (en) 2013-03-21

Similar Documents

Publication Publication Date Title
US8115804B2 (en) Processing multiview video
US20070177671A1 (en) Processing multiview video
JP2023085341A (en) Efficient multi-view coding using estimation and update of depth map
DK2777283T3 (en) Effective prediction using partition encoding
KR20210032535A (en) Method and apparatus for processing video signals based on inter prediction
US20140192889A1 (en) Method and an apparatus for decoding a video signal
US20220182614A1 (en) Method and apparatus for processing video signal
WO2021244935A1 (en) A method, an apparatus and a computer program product for video encoding and video decoding
JP2010525724A (en) Method and apparatus for decoding / encoding a video signal
EP2243297A1 (en) Methods and apparatus for implicit block segmentation in video encoding and decoding
USRE44680E1 (en) Processing multiview video
WO2023202956A1 (en) Video coding and decoding
KR20080029944A (en) A method and apparatus for processing a video signal

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application
WWE Wipo information: entry into national phase

Ref document number: 2008550241

Country of ref document: JP

WWE Wipo information: entry into national phase

Ref document number: 200780003083.8

Country of ref document: CN

NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: 2007700952

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 1020087019748

Country of ref document: KR