Fixes: out of array access
Fixes: 19892/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_AGM_fuzzer-5707525924323328
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
s->dct = avctx->codec_tag != MKTAG('A', 'G', 'M', '4') &&
avctx->codec_tag != MKTAG('A', 'G', 'M', '5');
+ if (!s->rgb && !s->dct) {
+ if ((avctx->width & 1) || (avctx->height & 1))
+ return AVERROR_INVALIDDATA;
+ }
+
avctx->idct_algo = FF_IDCT_SIMPLE;
ff_idctdsp_init(&s->idsp, avctx);
ff_init_scantable(s->idsp.idct_permutation, &s->scantable, ff_zigzag_direct);