avcodec/tiff: do not allow bpp 40 with undefined pixel formats

Fixes: Out of array access, assertion failure
Fixes: 13851/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_TIFF_fuzzer-5754570929602560
Fixes: 13869/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_TIFF_fuzzer-5695998313103360
Fixes: 13873/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_TIFF_fuzzer-5077273645940736
Fixes: 13874/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_TIFF_fuzzer-5756396757319680
Fixes: 13877/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_TIFF_fuzzer-5741026750234624
Fixes: 13880/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_TIFF_fuzzer-5649148809838592
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Carl Eugen Hoyos <ceffmpeg@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

diff --git a/libavcodec/tiff.c b/libavcodec/tiff.c
index 29a80a6892261e05736351c5435b856fb20279e7..3130eb80482ca4949a2899a44259b70f58cc5f0e 100644 (file)
--- a/libavcodec/tiff.c
+++ b/libavcodec/tiff.c
@@ -830,6 +830,11 @@ static int init_image(TiffContext *s, ThreadFrame *frame)
     case 405:
         if (s->photometric == TIFF_PHOTOMETRIC_SEPARATED)
             s->avctx->pix_fmt = AV_PIX_FMT_RGBA;
+        else {
+            av_log(s->avctx, AV_LOG_ERROR,
+                "bpp=40 without PHOTOMETRIC_SEPARATED is unsupported\n");
+            return AVERROR_PATCHWELCOME;
+        }
         break;
     case 483:
         s->avctx->pix_fmt = s->le ? AV_PIX_FMT_RGB48LE  : AV_PIX_FMT_RGB48BE;