movenc: take packet dts shifting into mention in check_pkt

This FFmpeg-specific "fuzzer fix" was never perfect, but now it
stopped encoding of actual content with a big enough DTS shift.
This returns the function to its original state of results
before negative CTS offsets were added.

I remember dealing with this function before, but somehow had
forgotten about it during VDD. The test cases not tripping this
over also didn't help.

diff --git a/libavformat/movenc.c b/libavformat/movenc.c
index 9c3e6437d72e49f5fc6a81ac74b52c5377fbeec8..2838286141842ad2bf9bd97b40f391b7f0aecec8 100644 (file)
--- a/libavformat/movenc.c
+++ b/libavformat/movenc.c
@@ -4989,6 +4989,12 @@ static int check_pkt(AVFormatContext *s, AVPacket *pkt)
     } else
         ref = pkt->dts; // Skip tests for the first packet
 
+    if (trk->dts_shift != AV_NOPTS_VALUE) {
+        /* With negative CTS offsets we have set an offset to the DTS,
+         * reverse this for the check. */
+        ref -= trk->dts_shift;
+    }
+
     duration = pkt->dts - ref;
     if (pkt->dts < ref || duration >= INT_MAX) {
         av_log(s, AV_LOG_ERROR, "Application provided duration: %"PRId64" / timestamp: %"PRId64" is out of range for mov/mp4 format\n",