1 =============================================
2 SNOW Video Codec Specification Draft 20070103
3 =============================================
7 This Specification describes the snow syntax and semmantics as well as
9 The decoding process is precissely described and any compliant decoder
10 MUST produce the exactly same output for a spec conformant snow stream.
11 For encoding though any process which generates a stream compliant to
12 the syntactical and semmantical requirements and which is decodeable by
13 the process described in this spec shall be considered a conformant
19 MUST the specific part must be done to conform to this standard
20 SHOULD it is recommended to be done that way, but not strictly required
22 ilog2(x) is the rounded down logarithm of x with basis 2
29 u unsigned scalar value range coded
30 s signed scalar value range coded
43 if(keyframe || always_reset)
46 version u header_state
47 always_reset b header_state
48 temporal_decomposition_type u header_state
49 temporal_decomposition_count u header_state
50 spatial_decomposition_count u header_state
51 colorspace_type u header_state
52 chroma_h_shift u header_state
53 chroma_v_shift u header_state
54 spatial_scalability b header_state
55 max_ref_frames-1 u header_state
59 spatial_decomposition_type s header_state
61 mv_scale s header_state
63 block_max_depth s header_state
66 for(plane=0; plane<2; plane++){
67 quant_table[plane][0][0] s header_state
68 for(level=0; level < spatial_decomposition_count; level++){
69 quant_table[plane][level][1]s header_state
70 quant_table[plane][level][3]s header_state
78 for(y=0; y<block_count_vertical; y++)
79 for(x=0; x<block_count_horizontal; x++)
85 y_diff=cb_diff=cr_diff=0
87 if(level!=max_block_depth){
88 s_context= 2*left->level + 2*top->level + topleft->level + topright->level
89 leaf b block_state[4 + s_context]
91 if(level==max_block_depth || leaf){
92 intra b block_state[1 + left->intra + top->intra]
94 y_diff s block_state[32]
95 cb_diff s block_state[64]
96 cr_diff s block_state[96]
98 ref_context= ilog2(2*left->ref) + ilog2(2*top->ref)
100 ref u block_state[128 + 1024 + 32*ref_context]
101 mx_context= ilog2(2*abs(left->mx - top->mx))
102 my_context= ilog2(2*abs(left->my - top->my))
103 mvx_diff s block_state[128 + 32*(mx_context + 16*!!ref)]
104 mvy_diff s block_state[128 + 32*(my_context + 16*!!ref)]
125 this MUST NOT change within a bitstream
128 if 1 then the range coder contexts will be reset after each frame
130 temporal_decomposition_type
133 temporal_decomposition_count
136 spatial_decomposition_count
141 this MUST NOT change within a bitstream
144 log2(luma.width / chroma.width)
145 this MUST NOT change within a bitstream
148 log2(luma.height / chroma.height)
149 this MUST NOT change within a bitstream
155 maximum number of reference frames
156 this MUST NOT change within a bitstream
159 minimum of the number of available reference frames and max_ref_frames
160 for example the first frame after a key frame always has ref_frames=1
162 spatial_decomposition_type
164 0 is a 9/7 symmetric compact integer wavelet
165 1 is a 5/3 symmetric compact integer wavelet
167 stored as delta from last, last is reset to 0 if always_reset || keyframe
170 quality (logarthmic quantizer scale)
171 stored as delta from last, last is reset to 0 if always_reset || keyframe
174 stored as delta from last, last is reset to 0 if always_reset || keyframe
175 FIXME check that everything works fine if this changes between frames
179 stored as delta from last, last is reset to 0 if always_reset || keyframe
182 maximum depth of the block tree
183 stored as delta from last, last is reset to 0 if always_reset || keyframe
194 left and top are set to the respective blocks unless they are outside of
195 the image in which case they are set to the Null block
197 top-left is set to the top left block unless it is outside of the image in
198 which case it is set to the left block
200 if this block has no larger parent block or it is at the left side of its
201 parent block and the top right block is not outside of the image then the
202 top right block is used for top-right else the top-left block is used
206 level, ref, mx and my are 0
209 Motion Vector Prediction:
210 =========================
211 1. the motion vectors of all the neighboring blocks are scaled to
212 compensate for the difference of reference frames
214 scaled_mv= (mv * (256 * (current_reference+1) / (mv.reference+1)) + 128)>>8
216 2. the median of the scaled left, top and top-right vectors is used as
217 motion vector prediction
219 3. the used motion vector is the sum of the predictor and
220 (mvx_diff, mvy_diff)*mv_scale
224 ======================
225 the luma and chroma values of the left block are used as predictors
227 the used luma and chroma is the sum of the predictor and y_diff, cb_diff, cr_diff
236 Each sample in the LL0 subband is predicted by the median of the left, top and
237 left+top-topleft samples, samples outside the subband shall be considered to
238 be 0. To reverse this prediction in the decoder apply the following.
239 for(y=0; y<height; y++){
240 for(x=0; x<width; x++){
241 sample[y][x] += median(sample[y-1][x],
243 sample[y-1][x]+sample[y][x-1]-sample[y-1][x-1]);
246 sample[-1][*]=sample[*][-1]= 0;
247 width,height here are the width and height of the LL0 subband not of the final
258 Snow supports 2 wavelet transforms, the symmetric biorthogonal 5/3 integer
259 transform and a integer approximation of the symmetric biorthogonal 9/7
262 2D IDWT (inverse discrete wavelet transform)
263 --------------------------------------------
264 The 2D IDWT applies a 2D filter recursively, each time combining the
265 4 lowest frequency subbands into a single subband until only 1 subband
267 The 2D filter is done by first applying a 1D filter in the vertical direction
268 and then applying it in the horizontal one.
269 --------------- --------------- --------------- ---------------
270 |LL0|HL0| | | | | | | | | | | |
271 |---+---| HL1 | | L0|H0 | HL1 | | LL1 | HL1 | | | |
272 |LH0|HH0| | | | | | | | | | | |
273 |-------+-------|->|-------+-------|->|-------+-------|->| L1 | H1 |->...
274 | | | | | | | | | | | |
275 | LH1 | HH1 | | LH1 | HH1 | | LH1 | HH1 | | | |
276 | | | | | | | | | | | |
277 --------------- --------------- --------------- ---------------
282 1. interleave the samples of the low and high frequency subbands like
283 s={L0, H0, L1, H1, L2, H2, L3, H3, ... }
284 note, this can end with a L or a H, the number of elements shall be w
285 s[-1] shall be considered equivalent to s[1 ]
286 s[w ] shall be considered equivalent to s[w-2]
288 2. perform the lifting steps in order as described below
291 1. s[i] -= (s[i-1] + s[i+1] + 2)>>2; for all even i < w
292 2. s[i] += (s[i-1] + s[i+1] )>>1; for all odd i < w
294 \ | /|\ | /|\ | /|\ | /|\
295 \|/ | \|/ | \|/ | \|/ |
297 /|\ | /|\ | /|\ | /|\ |
298 / | \|/ | \|/ | \|/ | \|/
302 snows 9/7 Integer filter:
303 1. s[i] -= (3*(s[i-1] + s[i+1]) + 4)>>3; for all even i < w
304 2. s[i] -= s[i-1] + s[i+1] ; for all odd i < w
305 3. s[i] += ( s[i-1] + s[i+1] + 4*s[i] + 8)>>4; for all even i < w
306 4. s[i] += (3*(s[i-1] + s[i+1]) )>>1; for all odd i < w
308 \ | /|\ | /|\ | /|\ | /|\
309 \|/ | \|/ | \|/ | \|/ |
311 /|\ | /|\ | /|\ | /|\ |
312 / | \|/ | \|/ | \|/ | \|/
313 (| + (| + (| + (| + -1
314 \ + /|\ + /|\ + /|\ + /|\ +1/4
315 \|/ | \|/ | \|/ | \|/ |
316 + | + | + | + | +1/16
317 /|\ | /|\ | /|\ | /|\ |
318 / | \|/ | \|/ | \|/ | \|/
325 finetune initial contexts
326 spatial_decomposition_count per frame?
328 try to use the wavelet transformed predicted image (motion compensated image) as context for coding the residual coefficients
329 try the MV length as context for coding the residual coefficients
330 use extradata for stuff which is in the keyframes now?
331 the MV median predictor is patented IIRC
334 spatial_scalability b vs u (!= 0 breaks syntax anyway so we can add a u later)
345 GPL + GFDL + whatever is needed to make this a RFC