When performing a comparison of a signed int and an unsigned int, the
signed int is first converted to an unsigned int, so that negative
values are being treated as big, positive values. This can become a
problem in an overread check, namely when an overread already happened.
So change the type of the variable containing the amount of bits that
need to be left to signed.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
The maximum allowed index for an array access is FF_ARRAY_ELEMS - 1; yet
the current code allowed FF_ARRAY_ELEMS. This wasn't dangerous in practice,
as parameter sets with invalid ids were already filtered out during
reading.
Found via PVS-Studio (see ticket #8156).
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Up until now, read_frame_internal always initialized the packet it
received. But since the recent changes to ff_read_packet, this is no
longer needed: If the parsing queue is initially empty upon entering
read_frame_internal, the packet will now either contain content upon
success or be blank upon failure of ff_read_packet. If the parsing
queue is initially not empty, the packet will be overwritten with the
oldest one from the parsing queue.
Similarly, it is unnecessary to initialize ret in read_frame_internal.
In parse_packet, it is easily possible to only initialize the packet
used as temporary storage for the output if said packet is used at all;
furthermore, this packet doesn't need to be zero-initialized, because
av_init_packet will initialize every field except size and data and
those fields will be set by av_parser_parse2.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Up until now, parse_packet() used a stack packet in case the stream is
flushed. But using such a packet is unnecessary as there is an AVPacket
readily available, it just needs to be used. Whether flushing is intended
or not will now be signalled by an explicit parameter rather than by
whether the packet parameter is NULL. This removes a few checks in
parse_packet(), gets rid of the initialization of the stack packet and
also reduces usage of sizeof(AVPacket) in libavformat.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Up until now, read_frame_internal in avformat/utils.c uses a spare
packet on the stack that serves no real purpose: At no point in this
function is there a need for another packet besides the packet destined
for output:
1. If the packet doesn't need a parser, but is output as is, the content
of the spare packet (that at this point contains a freshly read packet)
is simply copied into the output packet (via simple assignment, not
av_packet_move_ref, thereby confusing ownership).
2. If the packet needs parsing, the spare packet will be reset after
parsing and any packets resulting from the packet read will be put into
a packet list; the output packet is not used here at all.
3. If the stream should be discarded, the spare packet will be
unreferenced; the output packet is not used here at all either.
Therefore the spare packet and the copies can be removed in principle.
In practice, one more thing needs to be taken care of: If ff_read_packet
failed, the output packet was not affected, now it is. But given that
ff_read_packet returns a blank (as if reset via av_packet_unref) packet
on failure, there is no problem from this side either.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
When AVFMT_FLAG_GENPTS is set, av_read_frame would put a reference to a
packet in the packet list (via av_packet_ref) and then immediately
thereafter unreference the original packet. This has been changed to
move the reference instead.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Up until now, avformat_find_stream_info had a potential for memleaks:
When everything was fine, it read packets and (depending upon whether
AVFMT_FLAG_NOBUFFER was set) put them in a packet list or unreferenced
them when they were no longer needed. But upon failure, said packets
would leak if they were not already on the packet list. This patch fixes
this.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Since the recent changes to ff_packet_list_put, the source packet will
be automatically reset when the reference is moved to the packet list,
so that it is unnecessary to reinitialize the packet in the loops in
parse_packet and ff_read_packet; initializing once at the beginning is
enough.
This also fixes a potential, but currently unexisting problem: If the
raw packet buffer was initially not empty and probe_codec() failed,
then the packet returned would not be initialized. But given that
probe_codec() currently can't fail (always returns 0) this was not an
acute danger.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
avcodec/ituh263dec: Make the condition for the studio slice start code match between ff_h263_resync() and ff_mpeg4_decode_studio_slice_header()
If they mismatch an infinite loop can occur Fixes: Timeout (infinite loop) Fixes: 17043/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MPEG4_fuzzer-5695051748868096 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
avcodec/ralf: Fix integer overflow in decode_channel()
Fixes: signed integer overflow: -1094995519 * 64 cannot be represented in type 'int' Fixes: 17030/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_RALF_fuzzer-5640695838146560 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: left shift of 1073741824 by 1 places cannot be represented in type 'int' Fixes: 16976/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_VC1_fuzzer-4847262047404032 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
avcodec/vc1_pred: Fix invalid shifts in scaleforopp()
Fixes: left shift of negative value -2 Fixes: 16964/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_VC1IMAGE_fuzzer-5757853565976576 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
avcodec/vc1_block: Fix invalid shift with rangeredfrm
Fixes: left shift of negative value -7 Fixes: 16959/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_WMV3_fuzzer-5200360825683968 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: overflow in aspect ratio calculation Fixes: signed integer overflow: 393215 * 14594 cannot be represented in type 'int' Fixes: 15728/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_WMV3IMAGE_fuzzer-5661588893204480 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
avcodec/apedec: Fix several integer overflows in predictor_update_filter() and do_apply_filter()
Fixes: negation of -2147483648 cannot be represented in type 'int'; cast to an unsigned type to negate this value to itself Fixes: signed integer overflow: -14527961 - 2147483425 cannot be represented in type 'int' Fixes: 16380/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-5645957131141120 Fixes: 16968/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-5716169901735936 Fixes: 17074/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-5198710497083392 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
avcodec/hevc_cabac: Tighten the limit on k in ff_hevc_cu_qp_delta_abs()
Values larger would fail subsequent tests.
Fixes: signed integer overflow: 5 + 2147483646 cannot be represented in type 'int' Fixes: 16966/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_HEVC_fuzzer-5695709549953024 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
avfilter/vf_xbr: Fix left shift of negative number
Affected every usage of vf_xbr, e.g. the FATE-tests filter-2xbr,
filter-3xbr, filter-4xbr.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com> Reviewed-by: Paul B Mahol <onemda@gmail.com> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
avfilter/vf_hqx: Fix undefined left shifts of negative numbers
Affected every usage of this filter; in particular, it affected the
FATE-tests filter-2xbr, filter-3xbr and filter-4xbr.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com> Reviewed-by: Paul B Mahol <onemda@gmail.com> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
libavcodec/ffv1enc.c accessed an array of uint8_t [32] via array[0][j]
in order to loop over all the uint8_t in this array of arrays. Of course
this implied an out-of-bounds access for array[0] and UBSan complained
about this. So replace this with nested loops; furthermore, factor this
out into a function of its own to easily break out of the nested loops.
This affected the FATE-tests vsynth1-ffv1, vsynth1-ffv1-v3-yuv420p,
vsynth1-ffv1-v3-yuv422p10, vsynth1-ffv1-v3-yuv444p16,
vsynth1-ffv1-v3-bgr0, vsynth1-ffv1-ffv1-v3-rgb48 as well as the
corresponding vsynth2-*, vsynth3-* and the vsynth_lena-* tests.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
swscale/x86/swscale: Fix undefined left shifts of negative numbers
This affected many FATE-tests: The number of failing tests went down
from 663 to 344. (Both numbers exclude tests that failed because of
unaligned accesses in code that is inside #if HAVE_FAST_UNALIGNED.)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
avcodec/4xm: Check index in decode_i_block() also in the path where its not used.
Fixes: Infinite loop Fixes: signed integer overflow: 2147483644 + 16 cannot be represented in type 'int' Fixes: 16169/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_FOURXM_fuzzer-5662570416963584 Fixes: 16782/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_FOURXM_fuzzer-5743163859271680 Fixes: 17641/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_FOURXM_fuzzer-5711603562971136 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Reviewed-by: Paul B Mahol <onemda@gmail.com> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: Infinite loop Fixes: 17620/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ATRAC3_fuzzer-5086123012915200 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Reviewed-by: Paul B Mahol <onemda@gmail.com> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Reviewed-by: Paul B Mahol <onemda@gmail.com> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
avcodec/hcom: Check that there are dictionary entries
Fixes: out of array read Fixes: 17617/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_HCOM_fuzzer-5674970478280704 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Reviewed-by: Paul B Mahol <onemda@gmail.com> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
avcodec/fitsdec: Prevent division by 0 with huge data_max
Fixes: division by 0 Fixes: 15657/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_FITS_fuzzer-5738154838982656 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Reviewed-by: Paul B Mahol <onemda@gmail.com> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: Infinite loop Fixes: 17611/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ACELP_KELVIN_fuzzer-5765134928052224 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Reviewed-by: Paul B Mahol <onemda@gmail.com> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: out of array access Fixes: 17609/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SBC_fuzzer-5758729319874560 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Reviewed-by: Paul B Mahol <onemda@gmail.com> Reviewed-by: Moritz Barsnick <barsnick@gmx.net> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Since reading 4 bits always returns a value in the range [0, 15], the
check for vps_id >= HEVC_MAX_VPS_COUNT, where HEVC_MAX_VPS_COUNT = 16, is redundant.
Signed-off-by: Andriy Gelman <andriy.gelman@gmail.com> Signed-off-by: James Almer <jamrial@gmail.com>
The MPEG-1/2 decoder uses avpriv_find_start_code to search for start
codes and worked with the resulting start code before checking that it
is really a start code of a slice. In particular, if the picture is so
big that a slice_vertical_position_extension is present, it added the
slice_vertical_position_extension as if it had a slice. Then a left
shift is performed, without making sure that the value to be shifted is
nonnegative.
Afterwards the end result is checked, but even if a start code of a
non-slice has been found, it might pass these checks: If
slice_vertical_position_extension is present a start code <
SLICE_MIN_START_CODE can lead to a macroblock-row index that appears
valid. Furthermore, the left shift might make an invalid start code
appear valid by discarding the highest bit.
This has been fixed by checking directly after avpriv_find_start_code
has returned.
Fixes ticket #8162 (which is about the undefined left shifts).
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Zhong Li [Thu, 19 Sep 2019 20:45:26 +0000 (04:45 +0800)]
lavc/qsv: Fix MSDK initialization failure in system memory mode
MSDK does not create internal acceleration device on Linux,
So MFXVideoCORE_SetHandle() is necessary.
It has been added for ff_qsv_init_session_device().
But missed for ff_qsv_init_internal_session() due to commit 1f26a23 overwrited commit db89f45
avformat/utils: Move the reference to the packet list
Up until now, ff_packet_list_put had a flaw: When it moved a packet to
the list (meaning, when it ought to move the reference to the packet
list instead of creating a new one via av_packet_ref), it did not reset
the original packet, confusing the ownership of the data in the packet.
This has been done because some callers of this function were not
compatible with resetting the packet.
This commit changes these callers and fixes this flaw. In order to
indicate that the ownership of the packet has moved to the packet list,
pointers to constant AVPackets are used whenever the target of the
pointer might already be owned by the packet list.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com> Signed-off-by: James Almer <jamrial@gmail.com>
ff_read_packet had potential memleaks:
1. If av_packet_make_refcounted fails, it means that the packet is not
refcounted, but it could nevertheless carry side data and therefore
needs to be unreferenced.
2. If putting a packet on a packet list fails, it wasn't unreferenced.
Furthermore, read_frame_internal leaked a packet's (side) data if a
context update was required and failed.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com> Signed-off-by: James Almer <jamrial@gmail.com>
There is currently an ordinary check for this (which would lead to a
memleak), but given that no demuxer should ever return a packet with an
invalid stream_index it is more appropriate for this to be an assert.
FATE passes with this change.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com> Signed-off-by: James Almer <jamrial@gmail.com>