stream_out: rtp: don't use VLA for user controlled data

It should fix a possible invalid memory access

When streaming ogg-files via rtp, an ogg-file can trigger an invalid
write access using an overly long 'configuration' string.

The original code attemps to allocate space to hold the string on the stack
and hence, cannot verify if allocation succeeds. Instead, we now allocate the
buffer on the heap and return if allocation fails.

In detail, rtp_packetize_xiph_config allocates a buffer on the stack at (1) where
the size depends on the local variable 'len'. The variable 'len' is
calculated at (0) to be the length of a string contained in a specially
crafted Ogg Vorbis file, and therefore, it is attacker-controlled.

Signed-off-by: Jean-Baptiste Kempf <jb@videolan.org>

diff --git a/modules/stream_out/rtpfmt.c b/modules/stream_out/rtpfmt.c
index 17f347878f47debaad4055035e312a99ab9e0d0d..cf8635bdd95fbf7180cf5b73e094a66dd1ffa4f2 100644 (file)
--- a/modules/stream_out/rtpfmt.c
+++ b/modules/stream_out/rtpfmt.c
@@ -559,7 +559,11 @@ int rtp_packetize_xiph_config( sout_stream_id_sys_t *id, const char *fmtp,
     char *end = strchr(start, ';');
     assert(end != NULL);
     size_t len = end - start;
-    char b64[len + 1];
+
+    char *b64 = malloc(len + 1);
+    if(!b64)
+        return VLC_EGENERIC;
+
     memcpy(b64, start, len);
     b64[len] = '\0';
 
@@ -569,6 +573,7 @@ int rtp_packetize_xiph_config( sout_stream_id_sys_t *id, const char *fmtp,
     int i_data;
 
     i_data = vlc_b64_decode_binary(&p_orig, b64);
+    free(b64);
     if (i_data <= 9)
     {
         free(p_orig);