avcodec/4xm: Fix signed integer overflows in idct()

Fixes: signed integer overflow: 20242 * 121095 cannot be represented in type 'int'
Fixes: 15310/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_FOURXM_fuzzer-5737051745419264
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

diff --git a/libavcodec/4xm.c b/libavcodec/4xm.c
index 89120aa8fb8ac3186f91301ffc4048321451bb94..8382159bdec76efac9a7a47ed470592f7df21f5e 100644 (file)
--- a/libavcodec/4xm.c
+++ b/libavcodec/4xm.c
@@ -158,7 +158,7 @@ typedef struct FourXContext {
 #define FIX_1_847759065 121095
 #define FIX_2_613125930 171254
 
-#define MULTIPLY(var, const) (((var) * (const)) >> 16)
+#define MULTIPLY(var, const) ((int)((var) * (unsigned)(const)) >> 16)
 
 static void idct(int16_t block[64])
 {