avcodec/vp9: Fix stack-buffer overflow with VP9 VDPAU available

ccca62ef991f0a47dfa30c3e822d91294b8afe4c added new VP9 VDPAU profiles
and as a consequence AV_PIX_FMT_VDPAU can now be twice in the list of
pixel formats used for format negotiation by ff_thread_get_format(); yet
there is only one entry in said list reserved for VDPAU, leading to a
stack-buffer overflow. This commit fixes this by making sure that
AV_PIX_FMT_VDPAU will not occur twice in said list.

Fixes Coverity ticket 1468046.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

diff --git a/libavcodec/vp9.c b/libavcodec/vp9.c
index 8b89fd68e2467565b52c811e4701e9347cc75ffd..294059f863f67c82c65eb044bb31eea05495968d 100644 (file)
--- a/libavcodec/vp9.c
+++ b/libavcodec/vp9.c
@@ -207,9 +207,6 @@ static int update_size(AVCodecContext *avctx, int w, int h)
 
         switch (s->pix_fmt) {
         case AV_PIX_FMT_YUV420P:
-#if CONFIG_VP9_VDPAU_HWACCEL
-            *fmtp++ = AV_PIX_FMT_VDPAU;
-#endif
         case AV_PIX_FMT_YUV420P10:
 #if CONFIG_VP9_DXVA2_HWACCEL
             *fmtp++ = AV_PIX_FMT_DXVA2_VLD;