Fixes out of array access Fixes: aaa479088e6fb40b04837b3119f47b04/asan_heap-oob_e38c68_8576_9d653078b2470700e2834636f12ff557.tga Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Mats Peterson [Sun, 17 Jan 2016 21:27:52 +0000 (22:27 +0100)]
lavc/rawdec: Use AV_PIX_FMT_PAL8 for 1-bit raw QuickTime video
Match the use of AV_PIX_FMT_PAL8 for 1-bit QuickTime Animation in
lavc/qtrle. To reiterate, 1-bit video is not necessary black & white in
QuickTime, merely bi-level. The two colors can be any color. The palette,
either included in the sample description, or the default Macintosh
palette (black & white for 1-bit video) will be set in lavf/qtpalette.
See the QuickTime File Format Specification for details.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
avcodec/tiff: Check subsample & rps values more completely
Fixes out of array access Fixes: 83aedfb29af669c4d6e10f1bfad974d2/asan_heap-oob_1ab42fe_4984_9f6ec14462f8d8a00ea24b320572a963.tif Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
It seems the previous tables where calculated with 32bit integers ignoring
overflows.
Also check for the max qindex, the value is choosen so that the qfactor/offset
fit in int32.
Fixes: 070b7914fd5dfe8f93248bea71363410/asan_static-oob_c8d034_2764_258e20f4a3c79158aecddb61a833d756.drc
Fixes out of array reads
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Claudio Freire [Sun, 17 Jan 2016 02:02:41 +0000 (23:02 -0300)]
AAC encoder: TNS fixes on short windows
TNS was computing filter coefficients incorrectly for short windows
due to a few coefficient addressing bugs. Fixing them fixes lots of
instability with transients (short windows).
Claudio Freire [Sat, 16 Jan 2016 23:47:00 +0000 (20:47 -0300)]
AAC encoder: use signed coeffs when measuring IS energy
I/S energy, especially when it comes to phase cancellations,
needs to use signed coefficients as input, yet it was using
abs'd coefficients. That was a slight bug.
Claudio Freire [Sat, 16 Jan 2016 23:44:36 +0000 (20:44 -0300)]
AAC encoder: enforce SF delta in PNS and IS SFs
Make sure to properly enforce maximum scalefactor delta in PNS
and I/S, not as an absolute range but relative to the previous
and first scalefactor, the latter important since PNS uses an
implicit inicial scalefactor derived from the global gain, and
encoded in a different way, which makes a big difference in the
allowed range.
Fixes out of array read Fixes: 1b79b985cdf860ffa228c00ee5497051/signal_sigsegv_1f99d24_315_f0f1fdb451264b1138e9c24eb7d49181.wv Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
s->ac_val[0][0] is of type 'int16_t [16]', but points into a larger buffer.
Here it is used as base pointer to find the correct position in the
larger buffer by adding 's->block_index[n] * 16' and thus as 'int16_t *'.
This fixes clang's ubsan runtime error: index out of bounds for type
'int16_t [16]'
Fixes: test_case.mp4 Found-by: Tyson Smith <twsmith@mozilla.com> Reviewed-by: Michael Niedermayer <michael@niedermayer.cc> Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Mats Peterson [Tue, 12 Jan 2016 23:32:20 +0000 (00:32 +0100)]
lavf/mov: Confine 0x00000000 to raw/twos fourcc mapping to version 0 sample descriptions
Confine the 0x00000000 to 'raw '/'twos' fourcc mapping to old version 0
sound sample descriptions, since they are the only valid sample
descriptions for this type of mapping.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array accesses Fixes: 0cf176e6d3ab9fe924f39738e513f547/asan_generic_4a54aa_3431_aaa28be1cb32e307a9890cad06f84fba.avi Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array access Fixes: 0cf176e6d3ab9fe924f39738e513f547/asan_generic_4a54aa_3431_aaa28be1cb32e307a9890cad06f84fba.avi Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
avformat/rmdec: Check size in ivr_read_packet() before use
Fixes out of array access Fixes: asan_heap-oob_445b39_1741_d00eb645ab48eb2203b4a04a5b997103.ivr Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Aman Gupta [Wed, 13 Jan 2016 01:27:06 +0000 (17:27 -0800)]
lavc/ccaption_dec: clear all unused rows during rollup
Sometimes rollup captions can move around the screen. This fixes "ghost"
captions from below the current rollup area from continuing to be
captured when a rollup moves higher up on the screen.
Aman Gupta [Sat, 9 Jan 2016 03:01:22 +0000 (19:01 -0800)]
lavc/ccaption_dec: implement real_time option
This new mode is useful for realtime decoding of closed captions so they
can be display along with mpeg2 frames.
Closed caption streams contain two major types of captions:
- POPON captions, which are buffered off-screen and displayed
only after EOC (end of caption, aka display buffer)
- PAINTON/ROLLUP captions, which are written to the display as soon as
they arrive.
In a typical real-time eia608 decoder, commands like EOC (end of
caption; display buffer), EDM (erase display memory) and EBM (erase
buffered memory) perform their expected functions as soon as the
commands are processed. This is implemented in the real_time branches
added in this commit.
Before this commit, and in the !real_time branches after this commit,
the decoder cleverly implements its own version of the decoder which is
specifically geared towards buffered decoding. It does so by actively
ignoring commands like EBM (erase buffered memory), and then re-using
the non-display buffer to hold the previous caption while the new one is
received. This is the opposite of the real-time decoder, which uses the
non-display buffer to hold the new caption while the display buffer is
still showing the current caption.
In addition to ignoring EBM, the buffered decoder also has custom
implementations for EDM and EOC. An EDM (erase display memory) command
flushes the existing contents before clearing the screen, and EOC
similarly always flushes the active buffer (the previous subtitle)
before flipping buffers.
swscale/vscale: Check that 2 tap filters are bilinear before using bilinear code
Fixes: out of array reads Fixes: 07e8b9c5d348ccdf7add0f37de20cf6c/asan_heap-oob_27e8df7_6849_e56653f768070ec8cb52f587048444c2.mov Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This makes SWS more robust Fixes: 07650a772d98aa63b0fed6370dc89037/asan_heap-oob_27ddeaf_2657_2c81ff264dee5d9712cb3251fb9c3bbb.264 Fixes: out of array read Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Claudio Freire [Thu, 14 Jan 2016 03:38:22 +0000 (00:38 -0300)]
avcodec/aacenc_is: replace pow(x, 0.75) by x/sqrtf(sqrtf(x))
This is quite an accurate approximation; testing shows ~ 2ulp error in
the floating point result. Tested with FATE.
Alternatively, if one wants "full accuracy", one can use powf, or sqrt
instead of sqrtf. With powf, one gets 1 ulp error (theoretically should be 0, as
0.75 is exactly representable) on GNU libm, with sqrt, 0 ulp error.
Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
Abstracted into pos_pow34 utility function
Fixes out of array read Fixes: 04442da73d935b776d2236282588d4f9/signal_sigsegv_2625a69_8790_ae85ffc889070663319b3417ede777b0.mov Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
All MDCT outputs must be checked in case of 128point MDCTs Fixes: out of array read Fixes: 04442da73d935b776d2236282588d4f9/signal_sigsegv_2625a69_351_52ca6226eb83547a2d26e322ce84ed84.mov Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Claudio Freire [Sun, 10 Jan 2016 06:04:21 +0000 (03:04 -0300)]
AAC encoder: encode out-of-phase I/S efficiently
Use the ability to invert phase with ms_mask instead of changing
the codebook when possible, to avoid having to switch codebooks
if some bands are INTENSTY_BT and others are INTENSITY_BT2, since
usually a set ms_mask uses less bits that a codebook change. While
it may not always be a win (ie: if it causes an ms_mask bitmap
to be sent when it wouldn't have been otherwise), it's unlikely
since the ms_mask bitmap will almost always be there already for
M/S itself.
Claudio Freire [Fri, 8 Jan 2016 09:18:24 +0000 (06:18 -0300)]
AAC encoder: fix I/S relative error evaluation
The relative error between two encoding strategies is the simple
difference of rate-distortion values, and not the absolute
difference. An absolute measure would allow worsening of the
quantization error as well as improving.
Claudio Freire [Fri, 8 Jan 2016 08:31:32 +0000 (05:31 -0300)]
AAC encoder: various fixes in M/S coding
1. Fix sf_idx and band_type addressing to address only the first
subwindow in the group (others could hold garbage values)
2. Don't step on ms_mask when is_mask is set. I/S selection
already sets the ms_mask properly and shouldn't be overridden.
3. Use mid/sid cb/sf when computing coding error, as should be
since those are the cb/sfs that will eventually be set.
4. Fix distortion computation on multi-subwindow groups (was
subtracting the bits terms multiple times)
5. Clear ms_mask when one side uses PNS and the other doesn't.
When using PNS, ms_mask signals correlated noise, which can be
detected just like regular M/S detection, so we don't skip
noise bands, but when only one side uses PNS setting the flag
can confuse some encoders, so avoid that.
Claudio Freire [Fri, 8 Jan 2016 08:16:18 +0000 (05:16 -0300)]
AAC encoder: in IS, fix index of sf_idx, band_type
Correct addressing of sf_idx and band_type arrays in I/S code.
Both arrays don't guarantee valid values for subwindows at all
times, depending on which coder is being used, which could result
in reading of garbage values. Instead, only the first subwindow
in the window group has to be used, ie: use w*16+g instead of
(w+w2)*16+g
projects / ffmpeg / log