avformat/iff: Check data_size not overflowing int64
Fixes: Infinite loop Fixes: 25844/clusterfuzz-testcase-minimized-ffmpeg_DEMUXER_fuzzer-5660803318153216 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Reviewed-by: Peter Ross <pross@xvid.org> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Anton Khirnov [Fri, 5 Jun 2020 09:17:49 +0000 (11:17 +0200)]
lavu/buffer: add a convenience function for replacing buffers
A common pattern e.g. in libavcodec is replacing/updating buffer
references: unref old one, ref new one. This function allows simplifying
such code and avoiding unnecessary refs+unrefs if the references are
already equivalent.
avformat/argo_brp: allow v1.1 ASF streams to have a non-22050 sample rate in certain circumstances
It seems that in files where the BASF block isn't first, v1.1 ASF streams are
allowed to be non-22050. Either this format is really inconsistent, or
FX Fighter and Croc just ignored the sample rate field, requiring the v1.1
restriction in the first place.
This bumps the version to 1.2 in these streams so they're not "corrected".
Found in Alien Odyssey games files in:
./GRAPHICS/COMMBUNK/{{COMADD1,COMM2_{1,2,3E},COMM3_{2,3,4,5,6}},FADE{1,2}}.BRP
Signed-off-by: Zane van Iperen <zane@zanevaniperen.com>
This proved beneficial for performance: For the sample [1] the number
of decicycles in one decode call decreased from 155851561 to 108158037
for Clang 10 and from 168270467 to 128847479 for GCC 9.3. For x86-32
compiled with GCC 9.3 and run on an x64 Haswell the number increased
from 158405517 to 202215769, so that the cached bitstream reader is only
enabled if HAVE_FAST_64BIT is set. These values are the average of 10
runs each looping five times over the input.
The fraps decoder already checked for overreads manually (and errored
out in this scenario), yet it still enabled implicit checks, leading to
worse performance and more code size.
This commit disables the implicit bitstream reader checks. For the
sample [1] this improves performance from 195105896 to 155851561
decicycles for Clang 10 and from 222801887 to 168270467 decicycles when
compiled with GCC 9.3. These values are the average of 10 runs each
looping ten times over the input.
avcodec/utvideodec: Avoid qsort when creating Huffman tables
The Ut video format uses Huffman trees which are only implicitly coded
in the bitstream: Only the lengths of the codes are coded, the rest has
to be inferred by the decoder according to the rule that the longer
codes are to the left of shorter codes in the tree and on each level the
symbols are descending from left to right.
Because longer codes are to the left of shorter codes, one needs to know
how many non-leaf nodes there are on each level in order to know the
code of the next left-most leaf (which belongs to the highest symbol on
that level). The current code does this by sorting the entries to be
ascending according to length and (for entries with the same length)
ascending according to their symbols. This array is then traversed in
reverse order, so that the lowest level is dealt with first, so that the
number of non-leaf nodes of the next higher level is known when
processing said level.
But this can also be calculated without sorting: Simply count how many
leaf nodes there are on each level. Then one can calculate the number of
non-leaf nodes on each level iteratively from the lowest level upwards:
It is just half the number of nodes of the level below.
This improves performance: For the sample from ticket #4044 the amount
of decicycles for one call to build_huff() decreased from 1055489 to
446310 for Clang 10 and from 1080306 to 535155 for GCC 9.
Reviewed-by: Paul B Mahol <onemda@gmail.com> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
avcodec/utvideodec/enc: Fix edge case of creating Huffman table
The Ut Video format stores Huffman tables in its bitstream by coding
the length of a given symbol; it does not code the actual code directly,
instead this is to be inferred by the rule that a symbol is to the left
of every shorter symbol in the Huffman tree and that for symbols of the
same length the symbol is descending from left to right. With one
exception, this is also what our de- and encoder did.
The exception only matters when there are codes of length 32, because
in this case the first symbol of this length did not get the code 0,
but 1; this is tantamount to pretending that there is a (nonexistent)
leaf of length 32. This is simply false. The reference software agrees
with this [1].
Now that the HuffEntries are no longer sorted by the MagicYUV decoder,
their symbols are trivial: The symbol of the element with index i is i.
They can therefore be removed. Furthermore, despite the length of the
codes being in the range 1..32 bits, the actual value of the codes is
<= 4096 (for 12 bit content). The reason for this is that the longer
codes are on the left side of the tree, so that the higher bits of
these codes are simply zero. By using an uint16_t for the codes and
removing the symbols entry, the size of each HuffEntry is decreased from
eight to four, saving 16KB of stack space.
Reviewed-by: Paul B Mahol <onemda@gmail.com> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
avcodec/magicyuv: Avoid AV_QSORT when creating Huffman table
The MagicYUV format stores Huffman tables in its bitstream by coding
the length of a given symbol; it does not code the actual code directly,
instead this is to be inferred by the rule that a symbol is to the left
of every shorter symbol in the Huffman tree and that for symbols of the
same length the symbol is ascending from left to right.
Our decoder implemented this by first sorting the array containing
length and symbol of each element according to descending length and
for equal length, according to ascending symbol. Afterwards, the current
state in the tree got encoded in a variable code; if the next array entry
had length len, then the len most significant bits of code contained
the code of this entry. Whenever an entry of the array of length
len was processed, code was incremented by 1U << (32 - len). So two
entries of length len have the same effect as incrementing code by
1U << (32 - (len - 1)), which corresponds to the parent node of length
len - 1 of the two nodes of length len etc.
This commit modifies this to avoid sorting the entries before
calculating the codes. This is done by calculating how many non-leaf
nodes there are on each level of the tree before calculating the codes.
Afterwards every leaf node on this level gets assigned the number of
nodes already on this level as code. This of course works only because
the entries are already sorted by their symbol initially, so that this
algorithm indeed gives ascending symbols from left to right on every
level.
This offers both speed- as well as (obvious) codesize advantages. With
Clang 10 the number of decicycles for build_huffman decreased from 1561987 to 1228405; for GCC 9 it went from 1825096 decicyles to 1429921.
These tests were carried out with a sample with 150 frames that was
looped 13 times; and this was iterated 10 times. The earlier reference
point here is from the point when the loop generating the codes was
traversed in reverse order (as the patch reversing the order led to
performance penalties).
Reviewed-by: Paul B Mahol <onemda@gmail.com> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
avcodec/magicyuv: Fix edge case of building Huffman table
The MagicYUV format stores Huffman tables in its bitstream by coding
the length of a given symbol; it does not code the actual code directly,
instead this is to be inferred by the rule that a symbol is to the left
of every shorter symbol in the Huffman tree and that for symbols of the
same length the symbol is ascending from left to right. With one
exception, this is also what our decoder did.
The exception only matters when there are codes of length 32, because
in this case the first symbol of this length did not get the code 0,
but 1; e.g. if there were exactly two nodes of length 32, then they
would get assigned the codes 1 and 2 and a node of length 31 will get
the 31-bit code 1 which is a prefix of the 32 bit code 2, making the
Huffman table invalid. On the other hand, if there were only one symbol
with the length 32, the earlier code would accept this un-Huffman-tree.
Reviewed-by: Paul B Mahol <onemda@gmail.com> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
avcodec/magicyuv: Don't invert order unnecessarily
The MagicYUV decoder currently sets both the length and the symbol field
of an array of HuffEntries; hereby the symbol of the ith entry (0-based)
is just i. Then said array gets sorted so that entries with greater
length are at the end and entries with the same length are ordered so
that those with smaller symbols are at the end. Afterwards the newly
sorted array is traversed in reverse order. This commit instead inverts
the ordering and traverses the array in its ordinary order in order to
simplify understanding.
Reviewed-by: Paul B Mahol <onemda@gmail.com> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
avcodec/dxtory: Fix negative shift in dx2_decode_slice_410()
Fixes: left shift of negative value -768 Fixes: 25574/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DXTORY_fuzzer-6012596027916288 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Reviewed-by: Paul B Mahol <onemda@gmail.com> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
avcodec/dxtory: Fix negative shift in dxtory_decode_v1_410()
Fixes: left shift of negative value -256 Fixes: 25460/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DXTORY_fuzzer-5073252341514240 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Reviewed-by: Paul B Mahol <onemda@gmail.com> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: out of array read Fixes: 25455/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DXTORY_fuzzer-6327985731534848 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Reviewed-by: Paul B Mahol <onemda@gmail.com> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
avcodec/mobiclip: Move quantizer check into setup_qtables()
Fixes: shift exponent -2 is negative Fixes: 25683/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MOBICLIP_fuzzer-6434808492982272 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Reviewed-by: Paul B Mahol <onemda@gmail.com> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
avformat/flvdec: RtmpSampleAccess no longer breaks stream detection
Since release 4.2, FFmpeg fails to detect the correct streams in an RTMP
stream that contains a |RtmpSampleAccess AMF object prior to the
onMetaData AMF object. In the debug log it would show "[flv] Unknown
type |RtmpSampleAccess".
This functionality broke in commit d7638d8dfc3c4ffd0dc18a64937a5a07ed67b354
as unknown metadata packets now result in an opaque data stream, and the
|RtmpSampleAccess packet was an "unknown" metadata packet type.
With this change the RTMP streams are correctly detected when there
is a |RtmpSampleAccess object prior to the onMetaData object.
Signed-off-by: Peter van der Spek <p.vanderspek@bluebillywig.com> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Shaofei Wang [Tue, 11 Jun 2019 06:52:29 +0000 (02:52 -0400)]
libavcodec/vp8dec: fix the multi-thread HWAccel decode error
Fix the issue: https://github.com/intel/media-driver/issues/317
the root cause is update_dimensions will be called multple times
when decoder thread number is not only 1, but update_dimensions
call get_pixel_format in each decode thread will trigger the
hwaccel_uninit/hwaccel_init more than once. But only one hwaccel
should be shared with all decode threads.
in current context,
there are 3 situations in the update_dimensions():
1. First time calling. No matter single thread or multithread,
get_pixel_format() should be called after dimensions were
set;
2. Dimention changed at the runtime. Dimention need to be
updated when macroblocks_base is already allocated,
get_pixel_format() should be called to recreate new frames
according to updated dimension;
3. Multithread first time calling. After decoder init, the
other threads will call update_dimensions() at first time
to allocate macroblocks_base and set dimensions.
But get_pixel_format() is shouldn't be called due to low
level frames and context are already created.
In this fix, we only call update_dimensions as need.
Signed-off-by: Wang, Shaofei <shaofei.wang@intel.com> Reviewed-by: Jun, Zhao <jun.zhao@intel.com> Reviewed-by: Haihao Xiang <haihao.xiang@intel.com> Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
Jan Ekström [Fri, 18 Sep 2020 20:50:54 +0000 (23:50 +0300)]
avutil/pixfmt: improve definition of AVColorRange
As it was brought up that the current documentation leaves things
as specific to YCbCr only, ICtCp and RGB are now mentioned.
Additionally, the specifications on which these definitions of
narrow and full range are defined are mentioned.
This way, the documentation of AVColorRange should now match how
most people seem to read interpret it at this point, and thus
flagging RGB AVFrames as full range is valid not only according to
common sense, but also the enum definition.
The earlier code would first attempt to allocate two buffers, then
attempt to allocate an AVIOContext, using one of the new buffers I/O
buffer, then check the allocations. On success, a z_stream that is used
in the AVIOContext's read_packet callback is initialized afterwards.
There are two problems with this: In case the allocation of the I/O
buffer fails avio_alloc_context() will be given a NULL read buffer
with a size > 0. This works right now, but it is fragile. The second
problem is that the z_stream used in the read_packet callback is not
functional when avio_alloc_context() is allocated (it might be that
avio_alloc_context() might already fill the buffer in the future). This
commit fixes both of these problems by reordering the operations.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Jan Ekström [Sun, 20 Sep 2020 12:26:42 +0000 (15:26 +0300)]
avformat/movenc: implement writing of the btrt box
This is utilized by various media ingests to figure out the bit
rate of the content you are pushing towards it, so write it for
video, audio and subtitle tracks in case at least one nonzero value
is available. It is only mentioned for timed metadata sample
descriptions in QTFF, so limit it only to ISOBMFF (MODE_MP4) mode.
Updates the FATE tests which have their results changed due to the
20 extra bytes being written per track.
for some cases (for example, super resolution), the DNN model changes
the frame size which impacts the filter behavior, so the filter needs
to know the out frame size at very beginning.
Currently, the filter reuses DNNModule.execute_model to query the
out frame size, it is not clear from interface perspective, so add
a new explict interface DNNModel.get_output for such query.
dnn: put DNNModel.set_input and DNNModule.execute_model together
suppose we have a detect and classify filter in the future, the
detect filter generates some bounding boxes (BBox) as AVFrame sidedata,
and the classify filter executes DNN model for each BBox. For each
BBox, we need to crop the AVFrame, copy data to DNN model input and do
the model execution. So we have to save the in_frame at DNNModel.set_input
and use it at DNNModule.execute_model, such saving is not feasible
when we support async execute_model.
This patch sets the in_frame as execution_model parameter, and so
all the information are put together within the same function for
each inference. It also makes easy to support BBox async inference.
Guo, Yejun [Fri, 28 Aug 2020 04:51:44 +0000 (12:51 +0800)]
dnn: change dnn interface to replace DNNData* with AVFrame*
Currently, every filter needs to provide code to transfer data from
AVFrame* to model input (DNNData*), and also from model output
(DNNData*) to AVFrame*. Actually, such transfer can be implemented
within DNN module, and so filter can focus on its own business logic.
DNN module also exports the function pointer pre_proc and post_proc
in struct DNNModel, just in case that a filter has its special logic
to transfer data between AVFrame* and DNNData*. The default implementation
within DNN module is used if the filter does not set pre/post_proc.
Fixes: signed integer overflow: 2 * 2132811776 cannot be represented in type 'int' Fixes: 25722/clusterfuzz-testcase-minimized-ffmpeg_IO_DEMUXER_fuzzer-6221704077246464 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Reviewed-by: Paul B Mahol <onemda@gmail.com> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
avformat/vividas: Check for EOF in first loop in track_header()
Fixes: timeout (243sec -> a few ms) Fixes: 25716/clusterfuzz-testcase-minimized-ffmpeg_IO_DEMUXER_fuzzer-5764093666131968 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Reviewed-by: Paul B Mahol <onemda@gmail.com> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: signed integer overflow: 6000 * -2147483648 cannot be represented in type 'int' Fixes: 25700/clusterfuzz-testcase-minimized-ffmpeg_DEMUXER_fuzzer-6578316302352384 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
avformat/dashdec: Fix memleak on allocation error, avoid allocation
get_content_url() allocates two buffers for temporary strings and when
one of them couldn't be allocated, it simply returns, although one of
the two allocations could have succeeded and would leak in this
scenario. This can be fixed by avoiding one of the temporary buffers.
Reviewed-by: Steven Liu <lq@chinaffmpeg.org> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
1. Perform the necessary reindentations after the last few commits.
2. Adapt switches to the ordinary indentation style.
3. Now that the effective lifetimes of the variables containing
the freshly allocated strings used when parsing the representation
are disjoint, the variables can be replaced by a single variable.
Doing so has the advantage of making it more clear that these are
throwaway variables, hence it has been done.
Reviewed-by: Steven Liu <lq@chinaffmpeg.org> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
avformat/dashdec: Return early for unsupported representations
This allows to reduce the level of indentation for parsing the supported
representations (audio, video and subtitles). It also allows to avoid
some allocations and frees for unsupported representations.
Reviewed-by: Steven Liu <lq@chinaffmpeg.org> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
This commit removes two always-true checks as well as a dead default
case of a switch. The check when parsing manifests is always true,
because we now jump to the cleaning code in case the format of the
representation is unknown. The default case of the switch is dead,
because the type of the representation is already checked at the
beginning of parse_manifest_representation(). The check when reading
the header is dead, because we error out if an error happened before.
Reviewed-by: Steven Liu <lq@chinaffmpeg.org> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
avformat/dashdec: Fix memleaks on error to add representation to dynarray
Up until now, the DASH demuxer used av_dynarray_add() to add
audio/video/subtitles representations to arrays. Yet av_dynarray_add()
frees the array upon failure, leading to leaks of its elements;
furthermore, the element to be added leaks, too.
This has been fixed by using av_dynarray_add_nofree() instead and by
freeing the elements that could not be added to the list. Furthermore,
errors from this are now checked and returned.
Reviewed-by: Steven Liu <lq@chinaffmpeg.org> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
avformat/dashdec: Fix leak of representation languages
These languages are normally freed after having been added as metadata
to their respective AVStreams. Yet if one never reaches said point, they
leak. This can happen as a result of an error when reading the header or
as a result of refreshing the manifests.
Reviewed-by: Steven Liu <lq@chinaffmpeg.org> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
avformat/dashdec: Fix leak of string on error when parsing representation
The DASH demuxer currently extracts several strings at once from an xml
document before processing them one by one; these strings are allocated,
stored in local variables and need to be freed by the demuxer itself.
So if an error happens when processing one of them, all strings need to
be freed before returning. This has simply not been done, leading to
leaks.
A simple fix would be to add the necessary code for freeing; yet there is
a better solution: Avoid having several strings at the same time by
extracting a string, processing it and immediately freeing it. That way
one only has to free at most one string on error.
Reviewed-by: Steven Liu <lq@chinaffmpeg.org> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
avformat/dashdec: Fix leak of representation on error
If parsing a representation fails, it is not added to the list of
representations and is therefore not freed in dash_close(); it therefore
leaked in most error paths in parse_manifest_representation() (some
error paths had (incomplete) code for freeing). This commit fixes
freeing the representation in this case.
Reviewed-by: Steven Liu <lq@chinaffmpeg.org> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
avformat/dashdec, hls: Update correct pointer to AVDictionary
open_url() in the DASH as well in the hls demuxer share a common bug:
They modify an AVDictionary (i.e. set a new entry) given to them as
AVDictionary *, yet if this new entry leads to reallocation and
relocation of the AVDictionary, the caller's pointer will become
dangling, leading to use-after-frees. So pass an AVDictionary **.
(With the current implementation of AVDictionary the above can only
happen if the AVDictionary was empty initially (in which case the
new AVDictionary leaks); furthermore if the I/O is ordinary (i.e. opened
by avio_open2() or ffio_open_whitelist()), the dict is never empty (it
contains an rw_timeout entry from save_avio_options()). So this issue
could only happen if the caller sets a nondefault io_open callback, but
no AVIOContext (the AVFMT_FLAG_CUSTOM_IO flag won't be set in this
case). In case of the HLS demuxer, it was also necessary that setting
the "seekable" entry failed. Yet one should simply not rely on internals
of the AVDict API.)
Reviewed-by: Steven Liu <lq@chinaffmpeg.org> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
avformat/dashdec: Don't leave representation in inconsistent state on error
This currently doesn't cause any trouble, because the only caller did
not clean up the representation upon error at all; but fixing this is
a prerequisite for doing so.
Reviewed-by: Steven Liu <lq@chinaffmpeg.org> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
The code in question seems to have been copied from about 70 lines
above; yet the code here is only executed if some of the variables
(namely representation_segmenttemplate_node and fragment_template_node)
are NULL, so it makes no sense to check them for a child element.
Also remove a redundant resetting of a pointer to an AVFormatContext
after avformat_close_input() (which already sets the pointer to NULL).
Reviewed-by: Steven Liu <lq@chinaffmpeg.org> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
When using one of the AV_DICT_DONT_STRDUP_KEY/VAL flags, av_dict_set()
already frees the key/value on error, so that freeing it again would
lead to a double free.
Reviewed-by: Steven Liu <lq@chinaffmpeg.org> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
The AAX demuxer reads a 32bit number containing the amount of entries
of an array and stores it in an uint32_t. Yet when iterating over this
array, a loop counter of type int is used. This leads to undefined
behaviour if the amount of entries is not in the range of int; to avoid
this, it is generally good to use the same type for the loop counter as
for the variable it is compared to. This is done in one of the two loops
affected by this.
In the other loop, the undefined behaviour can begin even earlier: Here
the loop counter is multiplied by an uint16_t which can overflow as soon
as the loop counter is > 2^15. Using an unsigned type would avoid the
undefined behaviour, but truncation would still be possible, so use an
uint64_t.
Also use an uint32_t for a variable containing an index in said array.
projects / ffmpeg / log