Fixes: index 2304 out of bounds for type 'float [2304]'
Fixes: 16332/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_QDM2_fuzzer-5679142481166336
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
s->sub_sampling = s->fft_order - 7;
s->frequency_range = 255 / (1 << (2 - s->sub_sampling));
+ if (s->frame_size * 4 >> s->sub_sampling > MPA_FRAME_SIZE) {
+ avpriv_request_sample(avctx, "large frames");
+ return AVERROR_PATCHWELCOME;
+ }
+
switch ((s->sub_sampling * 2 + s->channels - 1)) {
case 0: tmp = 40; break;
case 1: tmp = 48; break;