Limin Wang [Wed, 1 Jul 2020 12:57:58 +0000 (20:57 +0800)]
FATE: fix colorbalance fate test failed on x86_32
floating point precision will cause rgb*max generate different value on
x86_32 and x86_64. have pass fate test on x86_32 and x86_64 by using
lrintf to get the nearest integral value for rgb * max before av_clip.
Reviewed-by: Paul B Mahol <onemda@gmail.com> Signed-off-by: Limin Wang <lance.lmwang@gmail.com>
Joe Ratterman [Thu, 28 May 2020 17:31:48 +0000 (10:31 -0700)]
Replace 'FontName' with 'Fontname' in the documentation.
This is the only use of 'FontName' with that capitalization, as both
source-code and tests use 'Fontname'. Having consistent capitalization
makes it easier to find the relevant source from the docs.
See these examples for other uses:
libavcodec/ass_split.c:68
tests/ref/fate/sub-cc:9
Guo, Yejun [Mon, 25 May 2020 08:16:22 +0000 (16:16 +0800)]
vf_dnn_processing.c: add dnn backend openvino
We can try with the srcnn model from sr filter.
1) get srcnn.pb model file, see filter sr
2) convert srcnn.pb into openvino model with command:
python mo_tf.py --input_model srcnn.pb --data_type=FP32 --input_shape [1,960,1440,1] --keep_shape_ops
See the script at https://github.com/openvinotoolkit/openvino/tree/master/model-optimizer
We'll see srcnn.xml and srcnn.bin at current path, copy them to the
directory where ffmpeg is.
I have also uploaded the model files at https://github.com/guoyejun/dnn_processing/tree/master/models
3) run with openvino backend:
ffmpeg -i input.jpg -vf format=yuv420p,scale=w=iw*2:h=ih*2,dnn_processing=dnn_backend=openvino:model=srcnn.xml:input=x:output=srcnn/Maximum -y srcnn.ov.jpg
(The input.jpg resolution is 720*480)
Also copy the logs on my skylake machine (4 cpus) locally with openvino backend
and tensorflow backend. just for your information.
$ time ./ffmpeg -i 480p.mp4 -vf format=yuv420p,scale=w=iw*2:h=ih*2,dnn_processing=dnn_backend=tensorflow:model=srcnn.pb:input=x:output=y -y srcnn.tf.mp4
…
frame= 343 fps=2.1 q=31.0 Lsize= 2172kB time=00:00:11.76 bitrate=1511.9kbits/s speed=0.0706x
video:1973kB audio:187kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.517637%
[aac @ 0x2f5db80] Qavg: 454.353
real 2m46.781s
user 9m48.590s
sys 0m55.290s
$ time ./ffmpeg -i 480p.mp4 -vf format=yuv420p,scale=w=iw*2:h=ih*2,dnn_processing=dnn_backend=openvino:model=srcnn.xml:input=x:output=srcnn/Maximum -y srcnn.ov.mp4
…
frame= 343 fps=4.0 q=31.0 Lsize= 2172kB time=00:00:11.76 bitrate=1511.9kbits/s speed=0.137x
video:1973kB audio:187kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.517640%
[aac @ 0x31a9040] Qavg: 454.353
real 1m25.882s
user 5m27.004s
sys 0m0.640s
Signed-off-by: Guo, Yejun <yejun.guo@intel.com> Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
Guo, Yejun [Mon, 25 May 2020 07:38:09 +0000 (15:38 +0800)]
dnn: add openvino as one of dnn backend
OpenVINO is a Deep Learning Deployment Toolkit at
https://github.com/openvinotoolkit/openvino, it supports CPU, GPU
and heterogeneous plugins to accelerate deep learning inferencing.
Please refer to https://github.com/openvinotoolkit/openvino/blob/master/build-instruction.md
to build openvino (c library is built at the same time). Please add
option -DENABLE_MKL_DNN=ON for cmake to enable CPU path. The header
files and libraries are installed to /usr/local/deployment_tools/inference_engine/
with default options on my system.
To build FFmpeg with openvion, take my system as an example, run with:
$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/deployment_tools/inference_engine/lib/intel64/:/usr/local/deployment_tools/inference_engine/external/tbb/lib/
$ ../ffmpeg/configure --enable-libopenvino --extra-cflags=-I/usr/local/deployment_tools/inference_engine/include/ --extra-ldflags=-L/usr/local/deployment_tools/inference_engine/lib/intel64
$ make
Here are the features provided by OpenVINO inference engine:
- support more DNN model formats
It supports TensorFlow, Caffe, ONNX, MXNet and Kaldi by converting them
into OpenVINO format with a python script. And torth model
can be first converted into ONNX and then to OpenVINO format.
see the script at https://github.com/openvinotoolkit/openvino/tree/master/model-optimizer/mo.py
which also does some optimization at model level.
- optimize at inference stage
It optimizes for X86 CPUs with SSE, AVX etc.
It also optimizes based on OpenCL for Intel GPUs.
(only Intel GPU supported becuase Intel OpenCL extension is used for optimization)
Signed-off-by: Guo, Yejun <yejun.guo@intel.com> Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
libavformat/mov: Fix memleaks when demuxing DV audio
The code for demuxing DV audio predates the introduction of refcounted
packets and when the latter was added, changes to the former were
forgotten. This meant that when avpriv_dv_produce_packet initialized the
packet containing the AVBufferRef, the AVBufferRef as well as the
underlying AVBuffer leaked; the actual packet data didn't leak: They
were directly freed, but not via their AVBuffer's free function.
https://samples.ffmpeg.org/ffmpeg-bugs/trac/ticket4671/dir1.tar.bz2
contains samples for this (enable_drefs needs to be enabled for them).
Moreover, errors in avpriv_dv_produce_packet were ignored; this has been
changed, too.
Furthermore, in the hypothetical scenario that the track has a palette,
this would leak, too, so reorder the code so that the palette code
appears after the DV audio code.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
avutil/avsscanf: Add () to avoid integer overflow in scanexp()
Fixes: signed integer overflow: 2147483610 + 52 cannot be represented in type 'int' Fixes: 23260/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_PBM_fuzzer-5187871274434560 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
avformat/utils: reorder duration computation to avoid overflow
Fixes: signed integer overflow: 8 * 9223372036854774783 cannot be represented in type 'long' Fixes: 23381/clusterfuzz-testcase-minimized-ffmpeg_DEMUXER_fuzzer-4818340509122560 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: out of array access Fixes: 23554/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APNG_fuzzer-4796622520451072.fuzz Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
avutil/common: Fix integer overflow in av_ceil_log2_c()
Fixes: left shift of 1913647649 by 1 places cannot be represented in type 'int' Fixes: 23572/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_WMALOSSLESS_fuzzer-5082619795734528 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
avcodec/wmalosslessdec: fix overflow with pred in revert_cdlms
Fixes: signed integer overflow: 2048 + 2147483646 cannot be represented in type 'int' Fixes: 23538/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_WMALOSSLESS_fuzzer-5227567073460224 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This patch removes a check which throws an error if
the log2 precinct width/height is 0. The standard allows
the first component to have 0 as the log2 width/height.
However, to ensure proper intialization of coding style,
an extra check has been added.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Limin Wang [Sun, 28 Jun 2020 12:46:19 +0000 (20:46 +0800)]
avcodec/mpeg12dec: Fix uninitialized data in fate-sub-cc-scte20
The issue is introduced in a705bcd763e344fa, please tested with below command line:
make V=1 fate-sub-cc-scte20 TARGET_EXEC="valgrind --error-exitcode=1"
Reported-by: Martin Storsjö <martin@martin.st> Reviewed-by: Martin Storsjö <martin@martin.st> Signed-off-by: Limin Wang <lance.lmwang@gmail.com>
Derek Buitenhuis [Tue, 23 Jun 2020 15:05:18 +0000 (16:05 +0100)]
lavf/movenc: Use a dynamic buffer when writing the mfra box
When doing streamed output, with e.g. +dash, if the mfra box ended
up being larger than the AVIOContext write buffer, the (unchecked)
seeking back to update the box size would silently fail and produce
an invalid mfra box.
This is similar to how other boxes are written in fragmented mode.
avformat/smoothstreaming: Don't write trailer of subcontext
Nothing written in avformat_write_trailer() for the submuxers will be
output anyway because the AVIOContexts used for actual output have been
closed before the call. Writing the trailer of the subcontext has probably
only been done in order to free the memory allocated by the submuxer.
And this job has been taken over by the deinit functions.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
avformat/mvdec: Fix integer overflow with billions of channels
Fixes: signed integer overflow: 1394614304 * 2 cannot be represented in type 'int' Fixes: 23491/clusterfuzz-testcase-minimized-ffmpeg_DEMUXER_fuzzer-5697377020411904 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
libavcodec/jpeg2000dec.c Fixed WRITE_FRAME and tile co-ordinates:
libopenjpeg2000 uses ceiling division while dividing tile
co-ordinates with the sample separation. Also, corrections
were made to the WRITE_FRAME macro.
Improves: p1_01.j2k and p1_07.j2k
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
avformat/avc, mxfenc: Avoid allocation of H264 SPS structure, fix memleak
Up until now, ff_avc_decode_sps would parse a SPS and return some
properties from it in a freshly allocated structure. Yet said structure
is very small and completely internal to libavformat, so there is no
reason to use the heap for it. This commit therefore changes the
function to return an int and to modify a caller-provided structure.
This will also allow ff_avc_decode_sps to return better error codes in
the future.
It also fixes a memleak in mxfenc: If a packet contained multiple SPS,
only the SPS structure belonging to the last SPS would be freed, the
other ones would leak when the pointer is overwritten to point to the
new SPS structure. Of course, without allocations there are no leaks.
This is Coverity issue #1445194.
Furthermore, the SPS structure has been renamed from
H264SequenceParameterSet to H264SPS in order to avoid overlong lines.
Reviewed-by: Tomas Härdin <tjoppen@acc.umu.se> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
After parsing the end timestamp of a WebVTT cue block, the current code
skips everything after the start of the timestamp that is not a \t, ' '
or \n and treats what is next as the start of a WebVTT cue settings list.
Yet if there is no such list, but a single \r, this will skip a part of
the cue payload (namely everything until the first occurence of \t, ' '
or \n) and treat what has not been skipped as the beginning of the
WebVTT cue settings list that extends until the next \r or \n (or the
end).
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
avformat/av1: Avoid using dynamic buffer when assembling av1c
Given that AV1 only has exactly one sequence header, it is unnecessary
to copy the content of said sequence header into an intermediate dynamic
buffer; instead the sequence header can be copied from where it is in
the input buffer.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Jun Zhao [Sun, 17 May 2020 04:10:05 +0000 (12:10 +0800)]
lavc/aac_ac3_parser: improve the raw AAC file bit rate calculation
Now we just use one ADTS raw frame to calculate the bit rate, it's
lead to a larger error when get the duration from bit rate, the
improvement cumulate Nth ADTS frames to get the average bit rate.
e,g used the command get the duration like:
ffprobe -show_entries format=duration -i fate-suite/aac/foo.aac
before this improvement dump the duration=2.173935
after this improvement dump the duration=1.979267
in fact, the real duration can be get by command like:
ffmpeg -i fate-suite/aac/foo.aac -f null /dev/null with time=00:00:01.97
avformat/microdvddec: skip malformed lines without frame number.
Fixes: signed integer overflow: 1 - -9223372036854775808 cannot be represented in type 'long' Fixes: 23490/clusterfuzz-testcase-minimized-ffmpeg_DEMUXER_fuzzer-5133490093031424 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Reviewed-by: Nicolas George <george@nsup.org> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
exwm [Fri, 19 Jun 2020 15:58:41 +0000 (15:58 +0000)]
avfilter/zoompan: add in_time variable
Currently, the zoompan filter exposes a 'time' variable (missing from docs) for use in
the 'zoom', 'x', and 'y' expressions. This variable is perhaps better named
'out_time' as it represents the timestamp in seconds of each output frame
produced by zoompan. This patch adds aliases 'out_time' and 'ot' for 'time'.
This patch also adds an 'in_time' (alias 'it') variable that provides access
to the timestamp in seconds of each input frame to the zoompan filter.
This helps to design zoompan filters that depend on the input video timestamps.
For example, it makes it easy to zoom in instantly for only some portion of a video.
Both the 'out_time' and 'in_time' variables have been added in the documentation
for zoompan.
Example usage of 'in_time' in the zoompan filter to zoom in 2x for the
first second of the input video and 1x for the rest:
zoompan=z='if(between(in_time,0,1),2,1):d=1'
V2: Fix zoompan filter documentation stating that the time variable
would be NAN if the input timestamp is unknown.
V3: Add 'it' alias for 'in_time. Add 'out_time' and 'ot' aliases for 'time'.
Minor corrections to zoompan docs.
Fixes: assertion failure Fixes: left shift of 1 by 31 places cannot be represented in type 'int' Fixes: 23264/clusterfuzz-testcase-minimized-ffmpeg_BSF_AV1_METADATA_fuzzer-6308429248593920 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
avfilter/vf_showpalette: Don't pretend disp_palette can fail
It can't fail, yet it returns an int and other code checks whether it
failed; yet if it did fail, an AVFrame would leak. One could of course
add an av_frame_free for this (that compilers could optimize away), yet
it is easier to simply stop pretending that disp_palette could fail.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
avformat/mov: Read attached pics directly into st->attached_pic
Given that av_get_packet returns a blank packet on error, the only
difference to the current approach (that uses intermediate AVPackets on
the stack) is that st->attached_pic will be properly initialized on error
(i.e. the timestamps are AV_NOPTS_VALUE) whereas right now st->attached_pic
is only zeroed.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Yet since 2215c39e94e01f57b3191a29e0e51d7e230daf49 (merged in commit 52c522c72090233edeeb0486a9bd8bee925a710a) this information is exported
via the properly documented AV_PKT_DATA_PARAM_CHANGE side data.
The merge commit therefore stopped exporting the 'F' sidedata; yet the
changes in the Nellymoser decoder (which are now dead code (and would
become dangerous if lots of new side data types were added)) have not
been removed. This commit does this.
Reviewed-by: Anton Khirnov <anton@khirnov.net> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
avformat/dashenc: Calculate average bitrate for adaptation sets in static manifest
If stream's bitrate is not specified:
- for static manifest: an average bitrate will be calculated and used,
- for dynamic manifest: first segment's bitrate will be calculated and used, as before,
for bandwidth setting in adaptation sets.
projects / ffmpeg / log