A method and apparatus for video encoding or decoding utilizing adaptive background residual prediction is disclosed. The present invention adaptively applies background residual prediction to a current block based on a selection decision. The coding block is split into one or more coding sub-blocks. A reference sub-block in a reference picture is located for a current coding sub-block of the current coding block according to a motion vector associated with the current coding block. A background reference sub-block in a background picture is located for the reference sub-block, where the background reference sub-block is at a first co-located location as the reference sub-block. The method then selects a first predictor or a second predictor to encode or decode the current sub-block based on a selection decision. The first predictor corresponds to the reference sub-block, and the second predictor is derived according to the reference sub-block and the background picture.
Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
1. A method of video encoding or decoding utilizing adaptive background residual prediction, the method comprising: receiving input data associated with a current coding block in a current picture; splitting the current coding block into one or more coding sub-blocks; locating a reference sub-block in a reference picture for a current sub-block of the current coding block according to a motion vector associated with the current coding block; locating a background reference sub-block in a background picture for the reference sub-block, wherein the background reference sub-block is positioned at a first set of location coordinates (x 1 , y 1 ) in the background picture, and the reference sub-block is positioned at the same first set of location coordinates (x 1 , y 1 ) in the reference picture; and selecting a first predictor or a second predictor to encode or decode the current sub-block based on a selection decision, wherein the first predictor corresponds to the reference sub-block and the second predictor is derived according to the reference sub-block and the background picture.
A video encoder/decoder uses adaptive background residual prediction. The current coding block in the current picture is divided into sub-blocks. For each sub-block, a reference sub-block in a reference picture is located using a motion vector. Then, a background reference sub-block is located in a background picture, positioned at the same coordinates as the reference sub-block. A predictor is selected to encode or decode the current sub-block. The first predictor is simply the reference sub-block. The second predictor uses both the reference sub-block and the background picture.
2. The method of claim 1 , wherein the background picture is generated based on one or more decoded pictures.
The video encoder/decoder described in claim 1 generates the background picture using one or more previously decoded pictures. This means the background picture represents an approximation of the scene without moving objects, built from past frames.
3. The method of claim 1 , wherein the selection decision is derived based on the reference sub-block and the background reference sub-block.
In the video encoder/decoder using adaptive background residual prediction from claim 1, the selection between the first predictor (reference sub-block) and the second predictor (using the reference sub-block and background picture) is based on a comparison between the reference sub-block and the background reference sub-block. The similarity/dissimilarity between these blocks determines which predictor is used.
4. The method of claim 3 , wherein a sum of absolute differences (SAD) or a mean squared error (MSE) between the reference sub-block and the background reference sub-block is derived, and the first predictor is selected if the SAD or the MSE is greater than a threshold and the second predictor is selected if the SAD or the MSE is smaller than the threshold.
For the video encoder/decoder using adaptive background residual prediction as described in claim 3, the selection between predictors is based on the Sum of Absolute Differences (SAD) or Mean Squared Error (MSE) between the reference sub-block and the background reference sub-block. If the SAD/MSE is above a threshold, the first predictor (reference sub-block) is used. If it's below the threshold, the second predictor (using the reference sub-block and background picture) is used.
5. The method of claim 3 , wherein absolute differences between the reference sub-block and the background reference sub-block are derived, the first predictor is selected if a number of absolute differences exceeding a threshold is larger than a selected number and the second predictor is selected otherwise, and wherein the selected number corresponds to a non-negative integer.
In the video encoder/decoder using adaptive background residual prediction from claim 3, the selection is determined by comparing absolute differences between pixels in the reference sub-block and the background reference sub-block. If the number of absolute differences exceeding a set threshold is larger than a predefined number, the first predictor (reference sub-block) is selected; otherwise, the second predictor (using the reference sub-block and background picture) is chosen. This allows for selecting different prediction methods based on pixel-level variations between the reference and background.
6. The method of claim 1 , wherein the selection decision is indicated by a flag.
The video encoder/decoder that uses adaptive background residual prediction from claim 1 signals the selection decision (choosing between the two predictors) using a flag in the bitstream. This flag indicates which predictor was used for a specific sub-block.
7. The method of claim 1 , further comprising incorporating a syntax element in an encoder side or parsing the syntax element in a decoder side, wherein the syntax element indicates whether to enable adaptive background residual prediction.
The video encoder/decoder described in claim 1 includes a syntax element (a bit or group of bits) that signals whether adaptive background residual prediction should be enabled or disabled. The encoder includes this syntax element in the bitstream, and the decoder parses it to determine whether to use this prediction method.
8. The method of claim 7 , wherein the syntax element is incorporated in a picture level or a slice header level of a video bitstream to control selection of the adaptive background residual prediction for a respective picture or slice.
In the video encoder/decoder described in claim 7, the syntax element controlling adaptive background residual prediction is included either at the picture level or in the slice header of the video bitstream. This allows enabling/disabling adaptive background residual prediction for an entire picture or a slice within a picture.
9. The method of claim 7 , wherein the syntax element is incorporated in coded data for the current coding block to control selection of the adaptive background residual prediction for the current coding block.
In the video encoder/decoder described in claim 7, the syntax element controlling adaptive background residual prediction is included within the coded data for the current coding block. This provides fine-grained control, allowing the selection of adaptive background residual prediction on a block-by-block basis.
10. The method of claim 9 , wherein said incorporating the syntax element in the encoder side and said parsing the syntax element in the decoder side are skipped for the current coding block if the current coding block is coded using Merge mode or Skip mode.
In the video encoder/decoder described in claim 9, if the current coding block is coded using Merge mode or Skip mode, the syntax element indicating whether to enable adaptive background residual prediction for the current coding block is skipped in both the encoder and decoder. This reduces overhead by avoiding unnecessary signaling for blocks already efficiently coded.
11. The method of claim 9 , wherein said incorporating the syntax element in the encoder side and said parsing the syntax element in the decoder side are skipped for the current coding block if the current coding block is predicted using prediction units smaller than the current coding block.
In the video encoder/decoder described in claim 9, if the current coding block is predicted using prediction units that are smaller than the current coding block itself, the syntax element enabling or disabling adaptive background residual prediction for the current coding block is skipped in both the encoder and decoder. This prevents unnecessary signaling when a block is already being handled with more detailed prediction.
12. The method of claim 9 , wherein said incorporating the syntax element in the encoder side and said parsing the syntax element in the decoder side are skipped for the current coding block if width or height of the current coding block is equal to or smaller than a selected size, wherein the selected size corresponds to a positive integer.
In the video encoder/decoder described in claim 9, if the width or height of the current coding block is equal to or smaller than a certain size, the syntax element indicating whether adaptive background residual prediction should be used for the current coding block is skipped in both the encoder and decoder. This avoids the overhead of signaling for small blocks where the benefit of adaptive background residual prediction may be limited.
13. The method of claim 12 , wherein the selected size is eight or sixteen.
In the video encoder/decoder described in claim 12, the selected size, which determines whether the syntax element signaling adaptive background residual prediction is skipped for smaller blocks, is either eight or sixteen pixels. This means blocks of size 8x8 or 16x16 pixels or smaller won't have the flag.
14. The method of claim 1 , wherein width of the current sub-block is from 1 to picture width and height of the current sub-block is from 1 to picture height.
In the video encoder/decoder using adaptive background residual prediction as described in claim 1, the width of the current sub-block can range from 1 pixel up to the full picture width, and the height can range from 1 pixel up to the full picture height. This defines the possible dimensions of the sub-blocks used in the prediction process.
15. The method of claim 1 , wherein the second predictor corresponds to a linear combination of the reference sub-block, the background reference sub-block and a background current sub-block in the background picture, and wherein the background current sub-block is positioned at a second set of location coordinates (x 2 , y 2 ) in the background picture, and the current sub-block is positioned at the same second set of location coordinates (x 2 , y 2 ) in the current picture.
In the video encoder/decoder using adaptive background residual prediction as described in claim 1, the second predictor is a linear combination of the reference sub-block, the background reference sub-block, and a background current sub-block. The background current sub-block is located at the same coordinates as the current sub-block in the current picture, but in the background picture. This combines information from both reference and background frames to create a more accurate prediction.
16. The method of claim 15 , wherein the second predictor is set to a sum of a reference residual and the background current sub-block, and wherein the reference residual is derived by subtracting the background reference sub-block from the reference sub-block.
In the video encoder/decoder using adaptive background residual prediction from claim 15, the second predictor is calculated by adding a "reference residual" to the background current sub-block. The reference residual is computed by subtracting the background reference sub-block from the reference sub-block. This means that the prediction is the background in the current location, plus the *difference* between what you expected the background to be (background reference) and the actual reference.
17. The method of claim 1 , wherein the motion vector associated with the current coding block is derived in a picture domain or a background residual domain, wherein the current picture and the reference picture are used to derive the motion vector for the picture domain, and current background residual data and reference background residual data are used to derive the motion vector for the background residual domain, the current background residual data corresponds to first differences between the current picture and the background picture, and the reference background residual data corresponds to second differences between the reference picture and the background picture.
In the video encoder/decoder using adaptive background residual prediction as described in claim 1, the motion vector for the current coding block can be derived in two ways: either in the standard picture domain (using the current and reference pictures) or in a "background residual domain." For the background residual domain, motion estimation is performed using the difference between the current picture and the background picture, and the difference between the reference picture and the background picture.
18. An apparatus for video encoding or decoding utilizing adaptive background residual prediction, the apparatus comprising one or more electronic circuits configured to: receive input data associated with a current coding block in a current picture; split the current coding block into one or more coding sub-blocks; locate a reference sub-block in a reference picture for a current sub-block of the current coding block according to a motion vector associated with the current coding block; locate a background reference sub-block in a background picture for the reference sub-block, wherein the background reference sub-block is positioned at a first set of location coordinates (x 1 , y 1 ) in the background picture, and the reference sub-block is positioned at the same first set of location coordinates (x 1 , y 1 ) in the reference picture; and select a first predictor or a second predictor to encode or decode the current sub-block based on a selection decision, wherein the first predictor corresponds to the reference sub-block and the second predictor is derived according to the reference sub-block and the background picture.
An apparatus (video encoder/decoder) for video encoding or decoding uses adaptive background residual prediction. The hardware is configured to: divide the current coding block into sub-blocks. For each sub-block, locate a reference sub-block in a reference picture using a motion vector. Then, locate a background reference sub-block in a background picture at the same coordinates as the reference sub-block. Select a predictor to encode/decode the current sub-block. The first predictor is simply the reference sub-block. The second predictor uses both the reference sub-block and the background picture.
19. A method of video encoding or decoding utilizing adaptive background residual prediction, the method comprising: receiving input data associated with a current coding block in a current picture; splitting the current coding block into one or more coding sub-blocks; locating a reference sub-block in a reference picture for a current sub-block of the current coding block according to a motion vector associated with the current coding block; locating a background reference sub-block in a background picture for the reference sub-block, wherein the background reference sub-block is positioned at a first set of location coordinates (x 1 , y 1 ) in the background picture, and the reference sub-block is positioned at the same first set of location coordinates (x 1 , y 1 ) in the reference picture; locating a background current sub-block in a background picture, wherein the background current sub-block is positioned at a second set of location coordinates (x 2 , y 2 ) in the background picture, and the current sub-block is positioned at the same second set of location coordinates (x 2 , y 2 ) in the current picture; calculating a predictor for the current sub-block according to the reference sub-block, the background reference sub-block, and the background current sub-block; and encoding or decoding the current sub-block using the predictor.
A video encoder/decoder uses adaptive background residual prediction. The current coding block in the current picture is divided into sub-blocks. For each sub-block, a reference sub-block in a reference picture is located using a motion vector. Then, a background reference sub-block is located in a background picture, positioned at the same coordinates as the reference sub-block. A background current sub-block is also located in the background picture at the same coordinates as the current sub-block. A predictor is calculated using the reference sub-block, the background reference sub-block, and the background current sub-block, and the current sub-block is encoded/decoded using it.
20. The method of claim 19 , further comprising incorporating a syntax element in an encoder side or parsing the syntax element in a decoder side, wherein the syntax element indicates whether to enable adaptive background residual prediction.
The video encoder/decoder described in claim 19 includes a syntax element (a bit or group of bits) that signals whether adaptive background residual prediction should be enabled or disabled. The encoder includes this syntax element in the bitstream, and the decoder parses it to determine whether to use this prediction method.
21. The method of claim 19 , wherein the predictor is set to a sum of a reference residual and the background current sub-block, and wherein the reference residual is derived by subtracting the background reference sub-block from the reference sub-block.
In the video encoder/decoder described in claim 19, the predictor is calculated by adding a "reference residual" to the background current sub-block. The reference residual is computed by subtracting the background reference sub-block from the reference sub-block. This means that the prediction is the background in the current location, plus the *difference* between what you expected the background to be (background reference) and the actual reference.
22. The method of claim 19 , wherein the motion vector associated with the current coding block is derived in a picture domain or a background residual domain, wherein the current picture and the reference picture are used to derive the motion vector for the picture domain, and current background residual data and reference background residual data are used to derive the motion vector for the background residual domain, the current background residual data corresponds to first differences between the current picture and the background picture, and the reference background residual data corresponds to second differences between the reference picture and the background picture.
In the video encoder/decoder using adaptive background residual prediction as described in claim 19, the motion vector for the current coding block can be derived in two ways: either in the standard picture domain (using the current and reference pictures) or in a "background residual domain." For the background residual domain, motion estimation is performed using the difference between the current picture and the background picture, and the difference between the reference picture and the background picture.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
November 20, 2014
September 5, 2017
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.