lavu/tx: clip when converting table values to fixed-point

INT32_MAX (2147483647) isn't exactly representable by a floating point
value, with the closest being 2147483648.0. So when rescaling a value
of 1.0, this could overflow when casting the 64-bit value returned from
lrintf() into 32 bits.
Unfortunately the properties of integer overflows don't match up well
with how a Fourier Transform operates. So clip the value before
casting to a 32-bit int.

Should be noted we don't have overflows with the table values we're
currently using. However, converting a Kaiser-Bessel window function
with a length of 256 and a parameter of 5.0 to fixed point did create
overflows. So this is more of insurance to save debugging time
in case something changes in the future.
The macro is only used during init, so it being a little slower is
not a problem.

diff --git a/libavutil/tx_priv.h b/libavutil/tx_priv.h
index a3738f68bd43e876947271edbd15bbf973c6948a..0ace3e90dc6a809b6310c91be4ad22edef9661e2 100644 (file)
--- a/libavutil/tx_priv.h
+++ b/libavutil/tx_priv.h
@@ -85,7 +85,7 @@ typedef void FFTComplex;
         (dim)   = (int)(((accu) + 0x40000000) >> 31);                          \
     } while (0)
 
-#define RESCALE(x) (lrintf((x) * 2147483648.0))
+#define RESCALE(x) (av_clip64(lrintf((x) * 2147483648.0), INT32_MIN, INT32_MAX))
 
 #define FOLD(x, y) ((int)((x) + (unsigned)(y) + 32) >> 6)