avcodec/vorbisdec: Check parameters in vorbis_floor0_decode() before divide

Fixes: division by zero
Fixes: 16183/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_VORBIS_fuzzer-5688966782648320
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

diff --git a/libavcodec/vorbisdec.c b/libavcodec/vorbisdec.c
index a86d4c4c36636d6cf8ea6f81f7a920359b252f0e..793f0797378c5b01f73c041ac8cf119673df73a9 100644 (file)
--- a/libavcodec/vorbisdec.c
+++ b/libavcodec/vorbisdec.c
@@ -1182,6 +1182,9 @@ static int vorbis_floor0_decode(vorbis_context *vc,
                     q *= q;
                 }
 
+                if (p + q == 0.0)
+                    return AVERROR_INVALIDDATA;
+
                 /* calculate linear floor value */
                 q = exp((((amplitude*vf->amplitude_offset) /
                           (((1ULL << vf->amplitude_bits) - 1) * sqrt(p + q)))