Up until now, the SpeedHQ encoder called a wrong function for init:
void ff_init_uni_ac_vlc(const uint8_t huff_size_ac[256],
uint8_t *uni_ac_vlc_len);
Yet the first argument actually used is of type RLTable; the size of
said struct is less than 256 if the size of a pointer is four, leading
to an access beyond the end of the RLTable.
This commit fixes this by calling the actually intended function:
init_uni_ac_vlc() from mpeg12enc.c. It was intended to use this
function [1], yet doing so was forgotten when the patch was actually
applied.
The earlier code would not complain if the remaining size was one byte
short of the desired size; and the way it performed the check could run
into signed integer overflow.
Fixes: signed integer overflow: 9223372036854775807 + 1 cannot be represented in type 'long' Fixes: Timeout Fixes: 26434/clusterfuzz-testcase-minimized-ffmpeg_dem_MV_fuzzer-5752845451919360 Fixes: 26444/clusterfuzz-testcase-minimized-ffmpeg_dem_BINK_fuzzer-4697773380993024 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Reviewed-by: Michael Niedermayer <michael@niedermayer.cc> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Hendrik Leppkes [Sat, 5 Dec 2020 10:41:23 +0000 (11:41 +0100)]
avcodec: set AV_CODEC_CAP_CHANNEL_CONF on decoders which set their own channels
The decoders in this set either have a fixed channel count, or read it
from the bitstream, and thus do not require the channel count as
external information.
Anton Khirnov [Tue, 1 Dec 2020 18:32:00 +0000 (19:32 +0100)]
smvjpegdec: merge into mjpegdec
SMVJPEG stores frames as slices of a big JPEG image. The decoder is
implemented as a wrapper that instantiates a full internal MJPEG
decoder, then forwards the decoded frames with offset data pointers.
This is unnecessarily complex and fragile, not supporting useful decoder
capabilities like direct rendering.
Re-implement the decoder inside the MJPEG decoder, which is accomplished
by returning each decoded frame multiple times, setting cropping
information appropriately on each instance.
One peculiar aspect of the previous design is that since
- the smvjpeg decoder returns one frame per input packet
- there are multiple frames in each packets (the aformentioned slices)
the demuxer needs to return each packet multiple times.
This is now also eliminated - the demuxer now returns each packet
exactly once, with the duration set to the number of frames it decodes
to.
This also removes one of the last remaining internal uses of the old
video decoding API.
Anton Khirnov [Sun, 29 Nov 2020 16:40:14 +0000 (17:40 +0100)]
lavf/mux: rewrite guessing the packet duration
Factor out the code into a separate muxing-specific function.
Stop accessing the deprecated AVStream-embedded codec context, use the
average framerate (if specified) instead.
avcodec/snow: Use ff_thread_once() in ff_snow_common_init()
ff_snow_common_init() currently initializes static data every time it is
invoked; given that both the Snow encoder and decoder have the
FF_CODEC_CAP_INIT_THREADSAFE flag set, this can lead to data races (and
therefore undefined behaviour) even though all threads write the same
values. This commit fixes this by using ff_thread_once() for the
initializations.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Fixes: out of memory access Fixes: 27787/clusterfuzz-testcase-minimized-ffmpeg_dem_RPL_fuzzer-4743666463408128.fuzz Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Reviewed-by: Paul B Mahol <onemda@gmail.com> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: out of array access Fixes: 27424/clusterfuzz-testcase-minimized-ffmpeg_dem_VIVIDAS_fuzzer-5682070692823040 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Philip Langdale [Tue, 8 Dec 2020 00:33:29 +0000 (16:33 -0800)]
avcodec/libaom: Support monochrome encoding with libaom >= 2.0.1
Monochrome encoding with libaom was buggy for a long time, but this was
finally sorted out in libaom 2.0.1 (2.0.0 is almost there but was still
buggy in realtime mode).
We'll keep support for libaom 1.x around until the LTS distros that
include it are EOL (which is still a long time from now).
avformat/framecrcenc: Don't read after the end of side-data
Nothing guarantees that the size of side data containing a palette
is actually divisible by four (although it should be); but for
big-endian systems, an algorithm is used that presupposed this.
So switch to an algorithm that does not overread: It processes
four bytes at a time, but only if all of them are contained in
the side data.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
avcodec/wmadec: Reduce the size of tables used to initialize VLC
By switching from ff_init_vlc_sparse() to ff_init_vlc_from_lengths() one
can replace a table of codes of type uint16_t by a table of symbols of
type uint8_t, saving space.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
avcodec/aactab: Make AAC encoder and decoders actually init-threadsafe
Commit 1a29804558c13ef512d9ef73a9b0d782af4fa5f2 guarded several
initializations of static data in the AAC decoders with an AVOnce and
set the FF_CODEC_CAP_INIT_THREADSAFE flag, believing the former to be
sufficient for the latter. It wasn't, because several of these static
tables are shared with other components, so that there might be data
races if they are initialized from multiple threads. This affected
initializing the ff_sine_* tables as well as initializing the
ff_aac_pow*sf_tab tables (shared between both decoders and encoder) as
well as ff_aac_kbd_* tables (shared between encoder and floating point
decoder).
Commit 3d62e7a30fa552be52d12b31e3e0f79153aff891 set the
FF_CODEC_CAP_INIT_THREADSAFE flag for the AAC encoder. More explicitly,
this commit used the same AVOnce to guard initializing ff_aac_pow*sf_tab
in the encoder and to guard initializing the static data of each
decoder; the ensuing catastrophe was "fixed" in commit ec0719264cb9a9d5cbaf225da48929aea24997a3 by using a single AVOnce
for each codec again. But the codec cap has not been removed and
therefore the encoder claimed to be init-threadsafe, but wasn't, because
of the same tables as above.
The ff_sine_* tables as well as ff_aac_pow*sf_tab tables have already
been fixed; this commit deals with the ff_aac_kbd_* tables, making the
encoder as well as the floating-point decoder init-threadsafe (the
fixed-point decoder is it already).
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
avcodec/aacdec, aactab: Move kbd tables to their only user
The floating point kbd tables for 120 and 960 samples are only used by
the floating point decoder whereas the fixed point kbd tables for 128
and 1024 samples are only used by the fixed point AAC decoder. So move
these tables to their only users. This ensures that they are not
accidentally used somewhere else without ensuring that initializing
these tables stays thread-safe (as it is now because the only place from
where they are initialized is guarded by an AVOnce).
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
avcodec/aacdec, sinewin: Move 120 and 960 point sine tables to aacdec
The floating point AAC decoder is the only user of these tables, so it
makes sense to move them there. Furthermore, initializing the ordinary
power-of-two sinetables is currently not thread-safe and if the 120- and
960-point sinetables were not moved, one would have to choose whether
to guard initializing these two tables with their own AVOnces or not.
Doing so would add unnecessary AVOnces as the AAC decoder already guards
initializing its static data by an AVOnce; not doing so would be fragile
if a second user of these tables were to be added.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
avcodec/mpegaudio_tablegen: Make exponential LUT shared
Both the fixed as well as the floating point mpegaudio decoders use
LUTs of type int8_t and uint32_t with 32K entries each; these tables
are completely the same, yet they are not shared. This commit makes
them shared. When both fixed as well as floating point decoders are
enabled, this saves 160KiB from the bss segment for a normal build
(translating into 160KiB less memory usage if both a shared as well as
a floating point decoder have actually been used) and 160KiB from the
binary for a build with hardcoded tables.
It also means that the code to create said LUTs is no longer duplicated
(for a normal build).
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
avcodec/mpegaudiodec: Hardcode tables to save space
The csa_tables (which always consist of 32 entries of four byte each,
but the type depends upon whether the decoder is fixed or
floating-point) are currently initialized once during decoder
initialization; yet it turns out that this is actually no benefit: The
code used to initialize these tables takes up 153 (fixed point) and 122
(floating point) bytes when compiled with GCC 9.3 with -O3 on x64, so it
is better to just hardcode these tables.
Essentially the same applies to the is_tables: They have a size of 128B
each and the code to initialize them occupies 149 (fixed point) resp.
140 (floating point) bytes. So hardcode them, too.
To make the origin of the tables clear, references to the code used to
create them have been added.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
avcodec/mpegaudio_tablegen: Don't inappropriately use static array
Each invocation of this function is only entered once, so using a static
array makes no sense (and given that the whole array is reinitialized at
the beginning of this function, it wouldn't even make sense if the
function were called multiple times).
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
The mpegaudio_tablegen header contains code to initialize several
tables; it is included in both the fixed as well as the floating point
mpegaudio decoders and some of these tables are only used by the fixed
resp. floating point decoders; yet both types are always initialized,
leaving the compiler to figure out that one of them is unused.
GCC 9.3 fails at this (even with -O3):
$ readelf -s mpegaudiodec_fixed.o|grep _float
28: 0000000000001660 32768 OBJECT LOCAL DEFAULT 4 expval_table_float
An actually unused table (expval_table_fixed/float) of size 32KiB is kept
and initialized (the reason for this is probably that this table is read
from, namely to initialize another table: exp_table_fixed/float; of course
the float resp. fixed tables are not used in the fixed resp. floating point
decoder).
Therefore #ifdef the unneeded tables away.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
avcodec/mpegaudiodec: Combine tables used to initialize VLCs
Up until now, there were several indiviual tables which were accessed
via pointers to them; by combining the tables, one can avoid said
pointers, saving space.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
avcodec/mpegaudiodec: Reduce the size of tables used to initialize VLCs
By switching from ff_init_vlc_sparse() to ff_init_vlc_from_lengths() one
can replace tables of codes of type uint16_t by tables of symbols of
type uint8_t; this saves about 1.3KB for both the fixed and floating
point decoders (if enabled).
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
avcodec/mv30: Reduce the size of tables used to initialize VLCs
By switching from ff_init_vlc_sparse() to ff_init_vlc_from_lengths() one
can remove the array of codes of type uint16_t here; given that the
symbols are the default ones (0,1,2,...), no explicit symbols table
needs to be added.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Expressions like array[get_vlc2()] can be optimized by using a symbols
table if the array is always the same for a given VLC. This requirement
is fulfilled for several VLCs used by the AAC decoders.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
avcodec/qdmc: Make tables used to initialize VLCs smaller
This is possible by switching to ff_init_vlc_from_lengths() which allows
to replace tables of codes of size uint16_t or uint32_t by tables of
symbols of size uint8_t; in case there already were symbols tables the
savings are even bigger.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
MagicYUV transmits its Huffman trees by providing the length of the code
corresponding to each symbol; then the decoder has to assemble the table
in such a way that (i) longer codes are to the left of the tree and (ii)
for codes of the same length the symbols are ascending from left to right.
Up until now the decoder did this as follows: It counted the number of
codes of each length and derived the first code of a given length via
(ii). Then the array of lengths is traversed a second time to create
the codes; there is one running counter for each length to do so. This
process creates a default symbol table (that is omitted).
This commit changes this as follows: Everything is indexed by the
position in the tree (with codes to the left first); given (i), we can
calculate the ranges occupied by the codes of each length; and with (ii)
we can derive the actual symbols of each code; the running counters for
each length are now used for the symbols and not for the codes.
Doing so allows us to switch to ff_init_vlc_from_lengths(); this has the
advantage that the codes table needs only be traversed once and that the
codes need not be sorted any more (right now, the codes that are so long
that they will be put into subtables need to be sorted so that codes
that end up in the same subtable are contiguous).
For a sample produced by our encoder (natural content, 4000 frames,
YUV420p, ten iterations, GCC 9.3) this decreased the amount of
decicycles for each call to build_huffman() from 1336049 to 1309401.
Notice that our encoder restricts the code lengths to 12 and our decoder
only uses subtables when the code is longer than 12 bits, so the sorting
that can be avoided does not happen at the moment. If one reduces the
decoder's tables to nine bits, the performance improvement becomes more
apparent: The amount of decicycles for build_huffman() decreased from 1165210 to 654055.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
avcodec/utvideodec: Avoid implicit qsort when creating Huffman tables
The Huffman trees used by Ut Video have two important characteristics:
(i) Longer codes are on the left of the tree and (ii) for codes of the
same length, the symbol is descending from left to right in the tree.
Therefore all the information that needs to be transmitted is how long
the code corresponding to a given symbol is; and this is also all that
is transmitted.
Before 341914495e5c2f60bc920b0c6f660e5948a47f5a, the decoder used qsort
to sort the (length, symbol) pairs by ascending length and for equal
lengths by ascending symbol. Since said commit, the decoder uses
a first pass over the lengths table to count how many symbols of each
length there are; with (i) one can then easily calculate the code of
the left-most code with a given length in the tree and from there one
can calculate the codes for all entries, using one running counter for
each possible length. This eliminated the explicit qsort in
build_huff().
Yet ff_init_vlc_sparse() sorts the table itself as it has to ensure that
all the entries that will be placed in the same subtable are contiguous.
The tables created now are non-contiguous (they are ordered by symbol
and codes of different length aren't ordered at all; only codes of the
same length are ordered according to (ii)).
This commit therefore modifies the algorithm used to automatically create
tables whose codes are sorted from left to right in the tree. The key to
do so is the observation that the counts obtained in the first pass can
be used to contain the range of the codes of each length in the second
pass: If counts[i] is the count of codes with length i, then the first
counts[32] codes are of length 32, the next counts[31] codes are of
length 31 etc. So one knows the index of the lowest symbol whose code
has length 32 (if any): It is counts[32] - 1 due to (ii), whereas the
index of the lowest symbol whose code has length 31 (if any) is
counts[32] + counts[31] - 1; the index of the second-to-lowest symbol of
length 32 (if existing) is counts[32] - 2 etc.
If one follows the algorithm outlined above, one can switch to
ff_init_vlc_from_lengths() which has no implicit qsort; it also means
that one can offload the computation of the codes.
This turned out to be beneficial for performance: For the sample from
ticket #4044 it decreased the decicycles spent on one call to
build_huff() from 508480 to 340688 (GCC 9.3, looping 10 times over the
file to get enough runs and then repeating this ten times); for another
sample (YUV420p, natural content, 5500 frames, also ten iterations)
the time went down from 382346 to 275533 decicycles.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Expressions like array[get_vlc2()] can be optimized by using a symbols
table if the array is always the same for a given VLC. This requirement
is fulfilled for several VLCs used by ATRAC3, therefore this commit
implements this. This comes without any additional costs when using
ff_init_vlc_from_lengths() as one can then remove the codes tables.
While at it, remove the arrays of pointers to the individual arrays and
put all lengths+symbol pairs in one big array.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
avcodec/intrax8: Reduce the size of tables used to initialize VLCs
By switching from ff_init_vlc_sparse() to ff_init_vlc_from_lengths() one
can replace an array of codes of type uint16_t with an array of symbols
of type uint8_t, saving space.
Also remove some more code duplication while at it.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
avcodec/vp3: Use symbols table for VP3 motion vectors
Expressions like array[get_vlc2()] can be optimized by using a symbols
table if the array is always the same for a given VLC. This requirement
is fulfilled for the VLC used for VP3 motion vectors. The reason it
hasn't been done before is probably that the array in this case
contained entries in the range -31..31; but this is no problem with
ff_init_vlc_from_lengths(): Just apply an offset of 31 to the symbols
before storing them in the table used to initialize VP3 motion vectors
and apply an offset of -31 when initializing the actual VLC.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
avcodec/vp3: Make tables used to initialize VLCs smaller
This is possible by switching to ff_init_vlc_from_lengths() because it
allows to replace codes of type uint16_t by symbols of type uint8_t; in
some cases (like here) it also allows to replace explicitly coded
codes by implicitly coded symbols.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
By switching to ff_init_vlc_from_lengths() one can apply both positive
as well as negative offsets for free; in this case it even saves space
because one replaces codes tables that don't fit into an uint8_t by
symbols tables that fit into an uint8_t or can even be completely
avoided as they are trivial.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
avcodec/atrac9dec: Don't create VLCs that are never used
The ATRAC9 decoder creates VLCs with parameters contained in
HuffmanCodebooks; some of these HuffmanCodebooks are empty and yet
VLCs (that were completely unused*) were created from them. Said VLC
contained a single table with 512 VLC_TYPE[2] entries, each of which
indicated that this is an invalid code. This commit stops creating said
VLCs.
*: read_coeffs_coarse() uses the HuffmanCodebook
at9_huffman_coeffs[cb][prec][cbi]. prec is c->precision_coarse[i] + 1
and every precision_coarse entry is in the 1..15 range after
calc_precision(), so prec is >= 2 (all codebooks with prec < 2 are
empty). The remaining empty codebooks are those with cb == 1 and cbi ==
0, yet this is impossible, too: cb is given by c->codebookset[i] and
this is always 0 if i < 8 (because those are never set to anything else
in calc_codebook_idx()) and cbi is given by at9_q_unit_to_codebookidx[i]
which is never zero if i >= 8.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
projects / ffmpeg / log