h264_mp4toannexb: Improve extradata overread checks

h264_mp4toannexb: Improve extradata overread checks

Currently during parsing the extradata, h264_mp4toannexb checks for
overreads by adding the size of the current unit to the current position
pointer and comparing this to the end position of the extradata. But
pointer comparisons and pointer arithmetic are only defined if it does not
exceed the object it is used on (one past the last element of an array
is allowed, too). In practice, this might lead to overflows. Therefore
the check has been changed to use bytestream2_get_bytes_left() which
means that the pointers get subtracted and the result gets compared to
the available size.

Furthermore, the error code has been fixed.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>