Yonglin Luo [Wed, 3 Jul 2019 02:05:36 +0000 (10:05 +0800)]
libavfilter/vf_colorspace.c: fix demarcation point of gamma linearize function
The linearize function (usually refered to EOTF) is the inverse of
delinearize function (usually referred to OETF). Demarcation point of
EOTF should be beta*delta, but the actual value used now in the source
code is beta.
For ITU Rec.709, they are 0.081 (0.018*4.5) and 0.018 respectively
(beta = 0.018 and delta = 4.5), and they correspond to pixel value 5
and 21 for an 8-bit image. Linearized result of pixel within that range
(5-21) will be different, but this commit will make linearize function
of the filter more accurate in the mathematical sense.
Signed-off-by: Yonglin Luo <vincenluo@tencent.com> Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
Guo, Yejun [Thu, 13 Jun 2019 05:30:38 +0000 (13:30 +0800)]
tools/python: add script to convert TensorFlow model (.pb) to native model (.model)
For example, given TensorFlow model file espcn.pb,
to generate native model file espcn.model, just run:
python convert.py espcn.pb
In current implementation, the native model file is generated for
specific dnn network with hard-code python scripts maintained out of ffmpeg.
For example, srcnn network used by vf_sr is generated with
https://github.com/HighVoltageRocknRoll/sr/blob/master/generate_header_and_model.py#L85
In this patch, the script is designed as a general solution which
converts general TensorFlow model .pb file into .model file. The script
now has some tricky to be compatible with current implemention, will
be refined step by step.
The script is also added into ffmpeg source tree. It is expected there
will be many more patches and community needs the ownership of it.
Another technical direction is to do the conversion in c/c++ code within
ffmpeg source tree. While .pb file is organized with protocol buffers,
it is not easy to do such work with tiny c/c++ code, see more discussion
at http://ffmpeg.org/pipermail/ffmpeg-devel/2019-May/244496.html. So,
choose the python script.
The subfile protocol treats an end of 0 as meaning "until EOF"; this got
implemented by simply setting the end to INT64_MAX. But seeking relative
to EOF or AVSEEK_SIZE seeking hasn't been adapted; the result is that
e.g. the duration of transport streams isn't correctly determined when
this option is used. This is fixed in this patch.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Bela Bodecs [Mon, 1 Jul 2019 02:24:21 +0000 (10:24 +0800)]
avformat/hlsenc: changing all filename length to MAX_URL_SIZE
Throughout hlsenc code, all filename related buffer lengths are set
hardcoded as 1024. This PATCH change it to general value as MAX_URL_SIZE
in internal.h
Reviewed-by: Steven Liu <lq@onvideo.cn> Signed-off-by: Bela Bodecs <bodecsb@vivanet.hu>
avcodec/hevc_ps: Fix integer overflow with num_tile_rows and num_tile_columns
Fixes: signed integer overflow: -2147483648 - 1 cannot be represented in type 'int' Fixes: 14880/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_HEVC_fuzzer-5130977304641536 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Reviewed-by: James Almer <jamrial@gmail.com> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
avcodec/apedec: Add k < 24 check to the only k++ case which lacks such a check
Fixes: 15255/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-5718831688843264 Fixes: left shift of 1 by 31 places cannot be represented in type 'int' Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: division by 0 Fixes: 15206/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_FITS_fuzzer-5657260212092928 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: out of array read Fixes: 15263/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_M101_fuzzer-5728999453491200 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: out of array read Fixes: 15261/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_LIBVORBIS_fuzzer-5764908467093504 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: 15271/clusterfuzz-testcase-minimized-ffmpeg_DEMUXER_fuzzer-5735262606327808 Fixes: signed integer overflow: -2147483648 - 8 cannot be represented in type 'int' Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Bela Bodecs [Fri, 28 Jun 2019 05:54:27 +0000 (13:54 +0800)]
avformat/hlsenc: temp_file usage for master playlist and vtt playlist
currently master playlist and subtitle playlist creation does not use
temporary files even when temp_file flag is set. Most of the use cases
it is not a problem because master playlist creation happens once on the
beginning of the whole process. But if master playlist is periodically
re-created because of master_pl_refresh_rate is set, non-atomic playlist
creation may cause problems in case of live streaming. This patch
correct this behavior by adding this functionality.
Reviewed-by: Steven Liu <lq@onvideo.cn> Signed-off-by: Bela Bodecs <bodecsb@vivanet.hu>
This avoids the need for each decoder separately having to handle this case
Fixes: shift exponent -100663046 is negative Fixes: out of array access Fixes: 15270/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_IFF_ILBM_fuzzer-5727829913763840 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
avcodec/videodsp_template: Fix overflow of addition
Fixes: addition of unsigned offset to 0x7f56fc26a9b6 overflowed to 0x7f56fc26a8be* Fixes: clusterfuzz-testcase-minimized-mediasource_MP4_AVC1_pipeline_integration_fuzzer-4917949056679936 Reported-by: Matt Wolenetz <wolenetz@google.com> Reviewed-by: Matt Wolenetz <wolenetz@google.com> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
avcodec/ffwavesynth: Check ts_end - ts_start for overflow
Fixes: signed integer overflow: 2314885530818453536 - -8926099139098304480 cannot be represented in type 'long' Fixes: 15259/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_FFWAVESYNTH_fuzzer-5764366093254656 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
avcodec/vc1dsp: Avoid undefined shifts in vc1_v_s_overlap_c / vc1_h_s_overlap_c
Fixes: left shift of negative value -13 Fixes: 15260/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_VC1_fuzzer-5702076048343040 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: left shift of negative value -4483 Fixes: 15256/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_TTA_fuzzer-5738691617619968 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: signed integer overflow: 214013 * 2531011 cannot be represented in type 'int' Fixes: 15254/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_QDMC_fuzzer-5698137026461696 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
avcodec/rv10: Avoid calculating undefined value that is unused
Fixes: shift exponent 64 is too large for 32-bit type 'int' Fixes: 15253/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_RV10_fuzzer-5671114300194816 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: division by zero Fixes: 15257/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_BINTEXT_fuzzer-5757352881422336 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: signed integer overflow: 3784 * 682038 cannot be represented in type 'int' Fixes: 15265/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_BINK_fuzzer-5088311799971840 Fixes: 15268/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_BINK_fuzzer-5666502344179712 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Reviewed-by: Reviewed-by: Peter Ross <pross@xvid.org> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
avcodec/bink: Fix integer overflow in unquantize_dct_coeffs()
Fixes: signed integer overflow: -3447 * 2883584 cannot be represented in type 'int' Fixes: 15265/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_BINK_fuzzer-5088311799971840 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Reviewed-by: Peter Ross <pross@xvid.org> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Commit 31f9032b added the audio_preload feature; its goal is to
interleave audio earlier than the rest. Unfortunately, it has never ever
worked, because the check for whether a packet should be interleaved
before or after another packet was completely wrong: When audio_preload
vanishes, interleave_compare_dts returns 1 if the new packet should be
interleaved earlier than the packet it is compared with and that is what
the rest of the code expects. But the codepath used when audio_preload is
set does the opposite.
Also fixes potential undefined behaviour (namely signed integer
overflow).
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
avcodec/motionpixels: Check for vlc error in mp_get_vlc()
Fixes: 15246/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MOTIONPIXELS_fuzzer-5168534407086080 Fixes: runtime error: index -1 out of bounds for type 'HuffCode [16]' Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
avcodec/loco: Limit lossy parameter so it is sane and does not overflow
Fixes: 15248/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_LOCO_fuzzer-5087440458481664 Fixes: signed integer overflow: 3 + 2147483647 cannot be represented in type 'int' Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: out of array access Fixes: 15166/clusterfuzz-testcase-minimized-ffmpeg_DEMUXER_fuzzer-5731062396747776 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: division by zero Fixes: 15210/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_FITS_fuzzer-5746033243455488 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
avcodec/aacpsdsp_template: Fix integer overflow in ps_hybrid_analysis_c()
Fixes: signed integer overflow: -1539565182 + -798086761 cannot be represented in type 'int' Fixes: 14807/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_AAC_FIXED_fuzzer-564925382682214 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
avcodec/truemotion2: Fix integer overflow in last loop in tm2_update_block()
Fixes: signed integer overflow: -1727985666 - 538976288 cannot be represented in type 'int' Fixes: 15031/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_TRUEMOTION2_fuzzer-5100228035739648 Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
The mov flavour of timed text uses the first two bytes of the packet as
a length field. And up until 11bef2fe said length field has been read
correctly in the mov2textsub bsf. But since then the next two bytes are
read as if they were the length field. This is fixed in this commit.
Reviewed-by: Philip Langdale <philipl@overt.org> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com> Signed-off-by: James Almer <jamrial@gmail.com>
avformat/matroskadec: Improve error/EOF checks III
Up until now, when an element was skipped, it was relied upon
ffio_limit to make sure that there is enough data available to skip.
ffio_limit itself relies upon the availability of the file's size. As
this needn't be available, the check has been refined: First one byte
less than intended is skipped, then another byte is read, followed by a
check of the error flags.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
avformat/matroskadec: Improve read error/EOF checks II
This commit fixes a number of bugs:
1. There was no check that no read error/EOF occured during
ebml_read_uint, ebml_read_sint and ebml_read_float.
2. ebml_read_ascii and ebml_read_binary did sometimes not forward
error codes; instead they simply returned AVERROR(EIO).
3. In particular, AVERROR_EOF hasn't been used and no dedicated error
message for it existed. This has been changed.
In order to reduce code duplication, the new error code NEEDS_CHECKING
has been introduced which makes ebml_parse check the AVIOContext's
status for errors.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
avformat/matroskadec: Improve read error/EOF checks I
ebml_read_num had a number of flaws:
1. The check for read errors/EOF was totally wrong. E.g. an EBML number
beginning with the invalid 0x00 would be considered a read error,
although it is just invalid data.
2. The check for read errors/EOF was done just once, after reading the
first byte of the EBML number. But errors/EOF can happen inbetween, of
course, and this wasn't checked.
3. There was no way to distinguish when EOF should be an error (because
the data has to be there) for which an error message should be emitted
and when it is not necessarily an error (namely during parsing of EBML
IDs). Such a possibility has been added and used.
All this was fixed; furthermore, the error messages for invalid EBML
numbers were improved and useless initializations were removed.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Up until now, webm_dash_manifest_cues used the return values of
ebml_read_num and ebml_read_length without checking for errors,
i.e. return values < 0. This has been changed.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
It is only necessary to zero the initial allocated memory used to store
the size of laced frames if the block used Xiph lacing. Otherwise no
unintialized data was ever used, so use av_malloc instead of av_mallocz.
Also use the correct type for the allocations.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Get rid of pr dependency and write the columns strictly
alphabetical without page size considerations (POSIX
specifies 66 lines as default).
Setting the page size via pr's -l option was considered,
but as there is issue #5680 which wants to avoid pr
mainly because it's not in busybox, we chose to replace
pr instead.
Before pr would attempt to write pages, thus if a page
boundary was reached, the output looked confusing as one
couldn't see there was a new page and the alphabetical
order was disrupted when scanning down one of the columns.
This change is based on a shell implementation submitted
before by Yejun.
Possible differences to the current version using pr:
1. pr implementations should truncate items to not overflow columns;
depending on how it's done not truncating shall be better IMHO.
2. pr implementations might balance columns differently;
we use minimum number of lines and might end up not
using all columns or might have lesser entries in the
last column(s)
3. we use spaces only for padding the columns; at least the GNU pr
version on my system also by default stuffs in tabs in addition
to a single space in between columns. I don't see that this
behaviour is demanded by POSIX, though I might be very well
overlooking things. Anyway for our use case I can't see a need
for having the additional tabs, or why it would be better compared
to padding with spaces only.
Fixes output for sizes with width < column width, too.
Bela Bodecs [Sat, 22 Jun 2019 13:55:54 +0000 (15:55 +0200)]
avformat/hlsenc: better error log message for var_stream_map content
When multiple variant streams are specified by var_stream_map option,
%v is expected either in the filename or in the last sub-directory name,
but only in one of them. When both of them contains %v string, current
error message only states half of the truth.
And even %v may appears several times inside the last sub-directory name
or in filename pattern.
This patch clarifies this in the log message and in the doc also.
avformat/matroskadec: Treat SimpleBlock as EBML_BIN
Up until now, the SimpleBlock was treated specially: It basically had
its own EBML category and it was also included in the BlockGroup EBML
syntax (although a SimpleBlock must not exist in a BlockGroup according
to the Matroska specifications). The latter fact also meant that
a MatroskaBlock's buffer was always unreferenced twice.
This has been changed: The type of a SimpleBlock is now an EBML_BIN.
The only way in which SimpleBlocks are still different is that they
share their associated structure with another unit (namely BlockGroup).
This is also used to unref the block: It is always unreferenced via the
BlockGroup syntax.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
avcodec/iff: finetune the palette size check in the mask case
Fixes: out of array access Fixes: 15381/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_IFF_ILBM_fuzzer-5668057826983936 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Reviewed-by: Peter Ross <pross@xvid.org> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: 15372/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_IFF_ILBM_fuzzer-5708881759567872 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Reviewed-by: Peter Ross <pross@xvid.org> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Before this commit, the Matroska muxer would read a block when required
to do so, parse the block, create and return the necessary AVPackets and
yet keep the blocks (in a dynamically allocated list), although they
aren't used at all any more. This has been changed. There is no list any
more and the block is immediately discarded after parsing.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Every new element of an EbmlList is zeroed initially in
ebml_parse_elem, so that in particular a SimpleBlock's duration is
initialized to zero. Therefore it is unnecessary to initialize this
field again (for SimpleBlocks) in matroska_parse_cluster_incremental.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
By default, the data_offset member of the AVFormatInternal of the
AVFormatContext associated with the MatroskaDemuxContext has not been
initialized explicitly by any Matroska-specific function, so that it was
initialized by default to the offset at the end of matroska_read_header,
i.e. usually to the offset of the length field of the first encountered
cluster. This meant that in case that the Matroska-specific seek-code
fails because there are no index entries for the target track a seek to
data_offset would be performed and ordinary parsing would start from
there which is nonsense: The length field would be treated as EBML ID and
(if the length field is not longer than four bytes (EBML numbers that
long are rejected as invalid EBML IDs)) whatever comes next would be
treated as its EBML size although it simply isn't.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
avformat/matroskadec: Get rid of cluster size field assumption
The earlier code relied on the length of clusters always being coded on
eight bytes as was the behaviour of libavformat's Matroska muxer until
recently. But given that our own Matroska muxer now (and mkvmerge from
time immemorial) creates files that don't conform to this assumption,
it is high time to get rid of this assumption.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
avformat/matroskadec: Remove non-incremental parsing of clusters
When the new incremental parser was introduced, the old parser was
kept, because the new parser was unable to handle the way SSA packets
are put into Matroska. But since 2014 (since c7d8dbad) this is no
longer needed, so that the old parser can be completely removed.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
When multiple variant streams are specified by var_stream_map option,
implementation assumes that each elementary stream is assigned only once
to any variant. But this is not checked. This patch makes this checking.
Signed-off-by: Bela Bodecs <bodecsb@vivanet.hu> Reviewed-by: Steven Liu<lq@onvideo.cn>
avformat/vividas: Fix invalid shift in decode_key()
Fixes: left shift of 1 by 31 places cannot be represented in type 'int' Fixes: 15118/clusterfuzz-testcase-minimized-ffmpeg_DEMUXER_fuzzer-5740230004441088 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
avformat/jacosubdec: Fix timeres to 1/100 units convertion overflow
Fixes: signed integer overflow: 44444442 * 100 cannot be represented in type 'int' Fixes: 15117/clusterfuzz-testcase-minimized-ffmpeg_DEMUXER_fuzzer-5164660531134464 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: left shift of 1 by 31 places cannot be represented in type 'int' Fixes: 15123/clusterfuzz-testcase-minimized-ffmpeg_DEMUXER_fuzzer-5738039235575808 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>