avcodec/atrac3: Check for huge block aligns

The largest documented frame size = block align is 1024 bytes
(https://wiki.multimedia.cx/index.php/ATRAC3)

Without a limit this can allocate arbitrary memory and trigger OOM
Fixes: OOM
Fixes: 18337/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ATRAC3_fuzzer-5763861478637568
Fixes: 18556/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ATRAC3AL_fuzzer-5646183334936576
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

diff --git a/libavcodec/atrac3.c b/libavcodec/atrac3.c
index dc19a3863e68aada4bc9246506a3d9e712961ac9..067aa23f1fa7ba993b0f831b51f98f9767646890 100644 (file)
--- a/libavcodec/atrac3.c
+++ b/libavcodec/atrac3.c
@@ -964,7 +964,7 @@ static av_cold int atrac3_decode_init(AVCodecContext *avctx)
         return AVERROR_INVALIDDATA;
     }
 
-    if (avctx->block_align >= UINT_MAX / 2 || avctx->block_align <= 0)
+    if (avctx->block_align > 1024 || avctx->block_align <= 0)
         return AVERROR(EINVAL);
 
     q->decoded_bytes_buffer = av_mallocz(FFALIGN(avctx->block_align, 4) +