avcodec/sgienc: return meaningful error code

Signed-off-by: Paul B Mahol <onemda@gmail.com>

avformat/mov: Check dimensions before setting aspect

Fixes division by 0
Fixes: asan_heap-uaf_143f420_728_cov_1441472032_HDV_0113.MP4
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

swresample/x86/rematrix_init: Check av_malloc* return codes, forward errors

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

swresample/x86/rematrix_init: Use av_mallocz_array()

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avcodec/vb: Check for av_mallocz() failure

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avfilter/f_sendcmd: consider it an error if there are no commands

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

doc/indevs: Docs for Blackmagic high bit depth video/audio

Reviewed-by: Timothy Gu <timothygu99@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

hevcdsp: ARM NEON optimized transforms

cherry picked from commit b153f55935969c794de4640f8d34e01c58e027ae

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

x86: hevc_mc: remove lea in EPEL_LOAD

The second parameter to the macro is always an immediate address,
so no lea is needed.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

x86: hevc_mc: fewer gpr autoloads for _v filters

In that case, it's just to load my, but mx/r3src is not used.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

doc/APIchanges: updated Data stream changes in APIChange

Signed-off-by: Anshul Maheshwari <er.anshul.maheshwari@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Merge remote-tracking branch 'cehoyos/master'

* cehoyos/master:
  Fix standalone compilation of the mov muxer with --disable-optimizations.

Merged-by: Michael Niedermayer <michaelni@gmx.at>

avfilter/hqdn3d: an invalid bit depth means a bug, not invalid read data

This code looks clumsy, and an assert would probably be more welcome.

avformat/internal: fix av_interleaved_write_frame typo reference

Fix standalone compilation of the mov muxer with --disable-optimizations.

Reported-by: Bernhard Döbler

avfilter/dctdnoiz: fix slice_h computation

ceilf() can only work if the reminder of the division is not 0.

This fixes memory errors with for instance:
  ffmpeg -f lavfi -i testsrc=s=800x500 -threads 3 -vf dctdnoiz -frames:v 1 -f null -

avformat/mxfenc: Remove unused variables

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

lavfi/subtitles: add force_style option

Signed-off-by: Eejya Singh <singh.eejya@gmail.com>
Signed-off-by: Clément Bœsch <u@pkh.me>

avdevice/decklink_common: Use defined(__APPLE__) instead of __APPLE__

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

x86/vp9dsp: fix clobbering of xmm6 on IDCT sse2 functions

Reviewed-by: Ronald S. Bultje <rsbultje@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>

avdevice/decklink_common: Fix Decklink for Mac

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avdevice/decklink: 10 Bit support for Decklink input device

Example to capture video clip at 1080i50 10 bit:
ffmpeg -bm_v210 1 -f decklink -i 'UltraStudio Mini Recorder@11' -acodec copy -vcodec copy output.avi

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

libavformat/mxfenc.c: added mxf opatom muxer

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Merge remote-tracking branch 'cehoyos/master'

* cehoyos/master:
  Support decoding yuv dpx images.
  Do not use -mdynamic-no-pic on OSX x86_64.

Merged-by: Michael Niedermayer <michaelni@gmx.at>

avcodec/utils: Assert that dst->progress is clear before ff_thread_ref_frame()

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avcodec/h264: Setup decoder to have matching reference to the EC code

Also move EC ref initialization to where the EC code is called.
Fixes out of array read
Fixes: asan_heap-uaf_143f420_142_20110805_112659_ch0.mkv
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avcodec/h264: Move COPY_PICTURE() to h264.h so it can be used from other parts of the h264 decoder

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Support decoding yuv dpx images.

Do not use -mdynamic-no-pic on OSX x86_64.

Fixes default compilation with icc.

avformat/omadec: Subtract headersize in timestamp calculation

Fixes pts/dts

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avformat/omadec: only compute timestamps based on bitrate if its set

Fixes division by zero

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Merge commit 'e352520e3ed7f08f19e63cd60e95da6bb6f037c1'

* commit 'e352520e3ed7f08f19e63cd60e95da6bb6f037c1':
  oma: Report a timestamp

Merged-by: Michael Niedermayer <michaelni@gmx.at>

Merge commit '8029af586fd5a5f2a0803001f9eff386e5545fe2'

* commit '8029af586fd5a5f2a0803001f9eff386e5545fe2':
  dxva2_hevc: properly fill the scaling list structure

See: b11994593ae4c3856db735800f24d38293838759
Merged-by: Michael Niedermayer <michaelni@gmx.at>

x86: lavc/hevc_mc: fix comments

The width parameter is now completely at the back, and actually
never used. This helps understanding the actual parameter list.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avcodec/faxcompr: use init_get_bits8()

Signed-off-by: Paul B Mahol <onemda@gmail.com>

hevc/sao: do in-place band filtering when possible

The copies are only needed when data must be restored, so skip them
when it must not be.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

x86: lavc: share more constant through defines

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

lavf/mlv: Only add streams that are supposed to contain frames.

Fixes ticket #4296.

Reviewed-by: Peter Ross

avcodec/twinvqdec: use init_get_bits8()

Signed-off-by: Paul B Mahol <onemda@gmail.com>

oma: Report a timestamp

Reported-By: jb@videolan.org
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

lavc/lossless_audiodsp: revert various commits

Their intent was to make the DSP work with wmalossless pro.
The later was fixed to work with the DSP.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

wmalossless: reset lms_update

It may contain garbage at the end, and zeroing allows using DSP
with longer loops.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

dxva2_hevc: properly fill the scaling list structure

The scaling list can be specified in either the SPS or PPS.
Additionally, compensate for the diagonal scan permutation applied
in the decoder.

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

pixdesc: Include more functions in FF_DISABLE_DEPRECATION_WARNINGS

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avutil/pixfmt: Clarify meaning of the A/X bit in RGB555 / remove wording about significant bit

Found-by:wm4
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avutil/pixfmt: Clarify the meaning of the alpha bits in rgb444 and similar formats

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avutil/pixfmt: Clarify the meaning of the "alpha" bit in rgb555/bgr555

Found-by: wm4
Reviewed-by: wm4 <nfxjfg@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avutil/pixfmt: Clarify the meaning of the alpha byte in RGB0 and similar formats

Found-by: wm4
Reviewed-by: wm4 <nfxjfg@googlemail.com>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Reviewed-by: Carl Eugen Hoyos <cehoyos@ag.or.at>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avcodec/ccaption_dec: Added Debug logs

Signed-off-by: Anshul Maheshwari <er.anshul.maheshwari@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avcodec/ccaption_dec: handle error from ass_sub api

Signed-off-by: Anshul Maheshwari <er.anshul.maheshwari@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avcodec/ccaption_dec: Added Roll up functionality

Signed-off-by: Anshul Maheshwari <er.anshul.maheshwari@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Merge commit '9295d10ea9d138462b7f67d16bf95ae9ca76aca6'

* commit '9295d10ea9d138462b7f67d16bf95ae9ca76aca6':
  fate: Add a test for AAC ELD480.

Merged-by: Michael Niedermayer <michaelni@gmx.at>

avcodec/h264_slice: assert that reinit does not occur after the first slice

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avcodec/h264_slice: ignore SAR changes in slices after the first

Fixes race condition and null pointer dereference
Fixes: signal_sigsegv_1472ac3_468_cov_2915641226_CABACI3_Sony_B.jsv
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avcodec/h264_slice: Check picture structure before setting the related fields

This might fix a hypothetical race condition

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

fate: Add a test for AAC ELD480.

The source is er_eld_2100np_48_ep0.mp4 from the official test set.

avcodec/h264_slice: Do not change frame_num after the first slice

Fixes potential race condition
Fixes: signal_sigsegv_1472ac3_468_cov_2915641226_CABACI3_Sony_B.jsv
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

x86: lavc: share more constants

Reviewed-by: "Ronald S. Bultje" <rsbultje@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Merge remote-tracking branch 'cehoyos/master'

* cehoyos/master:
  lavc/tscc: Make 32bit output opaque.

Merged-by: Michael Niedermayer <michaelni@gmx.at>

avutil/opt: Fix type used to access AV_OPT_TYPE_SAMPLE_FMT

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avutil/opt: Fix types used to access AV_OPT_TYPE_PIXEL_FMT

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

x86/hevc_mc: use aligned loads

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

x86/lossless_audiodsp: fix compilation with --disable-yasm

Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: James Almer <jamrial@gmail.com>

lavc/tscc: Make 32bit output opaque.

Found-by: wm4

nutdec: Remove unused variables

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

img2dec: Remove dead code

Reviewed-by: Reimar Döffinger <Reimar.Doeffinger@gmx.de>
Reviewed-by: Carl Eugen Hoyos <cehoyos@ag.or.at>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avcodec/avfft: Add simple self test

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

generate_wave_table: Add include for AVSampleFormat

Fixes warning in `make checkheaders`.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

audioconvert: Add missing include for FF_API_AUDIO_CONVERT

Fixes warning in `make checkheaders`.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

x86/hevc_sao: fix loading of RIP address

pb_eo must be handled as a rip relative address for MSVC64, so an
intermediate register is needed. Should fix link failures.

Suggested by Hendrik Leppkes and Christophe Gisquet.

Tested-By: Hendrik Leppkes <h.leppkes@gmail.com>
Reviewed-by: Christophe Gisquet <christophe.gisquet@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>

x86/hevc: use CLIPW macro when possible

Conflicts:
libavcodec/x86/hevc_mc.asm

Reviewed-by: James Almer <jamrial@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

x86: hevc_mc: use epel_hv 16-wide function

The epel_hv functions were still relying on only epel_hv 8-wide
being the maximum width instanciated.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

x86: hevc_mc: add AVX2 optimizations

before
33304 decicycles in luma_bi_1, 523066 runs, 1222 skips
38138 decicycles in luma_bi_2, 523427 runs, 861 skips
13490 decicycles in luma_uni, 516138 runs, 8150 skips
after
20185 decicycles in luma_bi_1, 519970 runs, 4318 skips
24620 decicycles in luma_bi_2, 521024 runs, 3264 skips
10397 decicycles in luma_uni, 515715 runs, 8573 skips

Conflicts:
libavcodec/x86/hevc_mc.asm
libavcodec/x86/hevcdsp_init.c

Reviewed-by: James Almer <jamrial@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avformat/rpl: check av_get_packet() for failure

Also make ret signed as it should be from start.

Signed-off-by: Paul B Mahol <onemda@gmail.com>

avcodec/h264: Be more strict on rejecting pps/sps changes

Fixes race condition
Fixes: signal_sigsegv_1472ac3_468_cov_2915641226_CABACI3_Sony_B.jsv
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avcodec/h264: Be more strict on rejecting pps_id changes

Fixes race condition
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avformat/thp: check av_get_packet() for failure

Signed-off-by: Paul B Mahol <onemda@gmail.com>

avcodec/h264_ps: More completely check the bit depths

Fixes out of array read
Fixes: asan_static-oob_30328b6_719_cov_3325483287_H264_artifacts_motion.h264
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avformat/tta: only check for header and seek table crc if requested

Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: James Almer <jamrial@gmail.com>

Revert "avcodec/x86/lossless_audiodsp: Make scalarproduct_and_madd_int16 prototypes more similar"

This reverts commit 3b4ffba3af968ae702e3a44f6b5f53445efc7363.

Unbreaks the SSSE3 code on mingw32

Conflicts:

libavcodec/x86/lossless_audiodsp.asm

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avcodec/x86/lossless_audiodsp: Move order&8 fallback into C code

This is simpler and more robust, and fixes mismatching XMM save restore
mismatches

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avcodec/vp3: pass correct context to av_log()

Fixes null pointer dereference
Fixes: signal_sigsegv_000_3694_cov_1050071691_theora.mkv
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avcodec/x86/lossless_audiodsp: Make scalarproduct_and_madd_int16 prototypes more similar

This is needed as the mmx code is used as fallback from the ssse3 code

Suggested-by: jamrial
Tested-by: wm4
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avcodec/hevc: drop redundant ifs()

Found-by: Clément Bœsch <u@pkh.me>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

hevcdsp: ARM NEON optimized deblocking filter

cherry picked from commit 1b9ee47d2f43b0a029a9468233626102eb1473b8

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

hevc: free sao buffers when receiving a new SPS

The buffer pointers would be otherwise overwritten, causing a
leak on e.g. PERSIST_RPARAM_A_RExt_Sony_1.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

vp9: assign PTS to visible instead of invisible frames

All the webm/vp9 files I have seen so far can have packets that contain
1 invisible and 1 visible frame. The vp9 parser separates them. Since
the invisible frame is always (?) the first sub-packet, the new packet
is assigned the PTS of the original packet, while the packet containing
the visible frame has no PTS.

This patch essentially reassigns the PTS from the invisible to the
visible frame.

Reviewed-by: "Ronald S. Bultje" <rsbultje@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

x86/hevcdsp: add ff_hevc_sao_edge_filter_{10,12}_{sse2,avx2}

Original x86 intrinsics code by Pierre-Edouard Lepere.
Yasm port, refactoring and optimizations by James Almer.

Benchmarks of BQTerrace_1920x1080_60_qp22.bin with an Intel Core i5-4200U

Width 32
342694 decicycles in sao_edge_filter_10, 16384 runs, 0 skips
29476 decicycles in ff_hevc_sao_edge_filter_32_10_ssse3, 16384 runs, 0 skips
13996 decicycles in ff_hevc_sao_edge_filter_32_10_avx2, 16381 runs, 3 skips

Width 64
581163 decicycles in sao_edge_filter_10, 8192 runs, 0 skips
59774 decicycles in ff_hevc_sao_edge_filter_64_10_ssse3, 8192 runs, 0 skips
28383 decicycles in ff_hevc_sao_edge_filter_64_10_avx2, 8191 runs, 1 skips

Signed-off-by: James Almer <jamrial@gmail.com>

x86/hevcdsp: add ff_hevc_sao_edge_filter_8_{ssse3,avx2}

Original x86 intrinsics code and initial yasm port by Pierre-Edouard Lepere.
Refactoring and optimizations by James Almer.

Benchmarks of BQTerrace_1920x1080_60_qp22.bin with an Intel Core i5-4200U

Width 32
158583 decicycles in edge, sao_edge_filter_8 runs, 0 skips
5205 decicycles in ff_hevc_sao_edge_filter_32_8_ssse3, 32767 runs, 1 skips
2942 decicycles in ff_hevc_sao_edge_filter_32_8_avx2, 32767 runs, 1 skips

Width 64
705639 decicycles in sao_edge_filter_8, 262144 runs, 0 skips
19224 decicycles in ff_hevc_sao_edge_filter_64_8_ssse3, 262111 runs, 33 skips
10433 decicycles in ff_hevc_sao_edge_filter_64_8_avx2, 262115 runs, 29 skips

Signed-off-by: James Almer <jamrial@gmail.com>

hevcdsp: remove compilation-time-fixed parameter from sao_edge_filter

The stride_src parameter is always 2 * MAX_PB_SIZE + FF_INPUT_BUFFER_PADDING_SIZE.

Signed-off-by: James Almer <jamrial@gmail.com>

avcodec/h264_refs: set last_pic_for_ec only if it has not been set previously

This ensures we do not loose the frame in case or multiple clears
Fixes out of array read
Fixes: asan_heap-oob_2fa47ea_2100_cov_1278768963_ff_add_pixels_clamped_mmx.m2ts
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Merge remote-tracking branch 'cehoyos/master'

* cehoyos/master:
  Fix compilation with --disable-protocol=udp

Merged-by: Michael Niedermayer <michaelni@gmx.at>

avcodec/alsdec: use init_get_bits8()

Signed-off-by: Paul B Mahol <onemda@gmail.com>

avcodec/cllc: use init_get_bits8()

Signed-off-by: Paul B Mahol <onemda@gmail.com>

Fix compilation with --disable-protocol=udp

avfilter/vf_fps: update frame drop comment

Found-by: ubitux
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

avformat/thp: Check av_get_packet() for failure not only for partial output

Fixes null pointer dereference
Fixes: signal_sigsegv_db2c1f_3108_cov_163322880_pikmin2_opening1_partial.thp
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

lavf/avc: add buffer padding to extradata allocation

ff_avc_write_annexb_extradata() allocates extradata, but don't add
FF_INPUT_BUFFER_PADDING_SIZE value

Signed-off-by: Lukasz Marek <lukasz.m.luki2@gmail.com>

avcodec/nvenc: De-compensate aspect ratio compensation of DVD-like content.

For reasons we are not privy to, nvidia decided that the nvenc encoder
should apply aspect ratio compensation to 'DVD like' content, assuming that
the content is not bt.601 compliant, but needs to be bt.601 compliant. In
this context, that means that they make the following, questionable,
assumptions:

1) If the input dimensions are 720x480 or 720x576, assume the content has
an active area of 704x480 or 704x576.

2) Assume that whatever the input sample aspect ratio is, it does not account
for the difference between 'physical' and 'active' dimensions.

From, these assumptions, they then conclude that they can 'help', by adjusting
the sample aspect ratio by a factor of 45/44. And indeed, if you wanted to
display only the 704 wide active area with the same aspect ratio as the full
720 wide image - this would be the correct adjustment factor, but what if you
don't? And more importantly, what if you're used to ffmpeg not making this kind
of adjustment at encode time - because none of the other encoders do this!

And, what if you had already accounted for bt.601 and your input had the
correct attributes? Well, it's going to apply the compensation anyway!
So, if you take some content, and feed it through nvenc repeatedly, it
will keep scaling the aspect ratio every time, stretching your video out
more and more and more.

So, clearly, regardless of whether you want to apply bt.601 aspect ratio
adjustments or not, this is not the way to do it. With any other ffmpeg
encoder, you would do it as part of defining your input paramters or
do the adjustment at playback time, and there's no reason by nvenc
should be any different.

This change adds some logic to undo the compensation that nvenc would
otherwise do.

nvidia engineers have told us that they will work to make this
compensation mechanism optional in a future release of the nvenc
SDK. At that point, we can adapt accordingly.

Signed-off-by: Philip Langdale <philipl@overt.org>
Reviewed-by: Timo Rothenpieler <timo@rothenpieler.org>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

swscale/utils: Limit filter shifting so as not to read from prior the array

Fixes out of array read
Fixes: asan_heap-oob_1fb2f9b_3780_cov_3984375136_usf.mkv
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>