rtsp: Fix infinite loop in listen mode with UDP transport

In listen mode with UDP transport, once the sender has sent
the TEARDOWN and closed the connection, poll will indicate that
one can read from the connection (indicating that the socket has
reached EOF and should be closed by the receiver as well). In this
case, parse_rtsp_message won't try to parse the command (because
it's no longer in state STREAMING), but previously just returned
zero.

Prior to f6161fccf8c5720ceac1ed1df8ba60ff8fed69f5, this caused
udp_read_packet to return zero, which is treated as EOF by
read_packet. But after that commit, udp_read_packet would continue
if parse_rtsp_message didn't return an explicit error code.

To keep the original behaviour from before that commit, more
explicitly return an error in parse_rtsp_message when in the wrong
state.

Fixes: #8840
Signed-off-by: Martin Storsjö <martin@martin.st>

diff --git a/libavformat/rtsp.c b/libavformat/rtsp.c
index 5d8491b74bace111d21ccd182e2df2a225a7800c..ad12f2ae98ef6010c785fc0ea1ef6a2dcefda5f8 100644 (file)
--- a/libavformat/rtsp.c
+++ b/libavformat/rtsp.c
@@ -1970,7 +1970,7 @@ static int parse_rtsp_message(AVFormatContext *s)
                 av_log(s, AV_LOG_WARNING,
                        "Unable to answer to TEARDOWN\n");
         } else
-            return 0;
+            return AVERROR_EOF;
     } else {
         RTSPMessageHeader reply;
         ret = ff_rtsp_read_reply(s, &reply, NULL, 0, NULL);