fix super rare underflow within FixedHypot if input is -2147483648

this has an insanely rare chance of happening in certain cases (train rush map has a few spots where this is somewhat likely)
abs does not really handle -2147483648 and its result is pretty much implementation defined/ub, since we compile with fwrapv flag it would just return -2147483648 which will cause this to try and access tantoangle at a negative index
we can just return INT32_MAX here since treating either as 2147483647 would return 2147483647 eitherways
This commit is contained in:
alufolie91 committed 2026-05-04 14:10:39 +02:00
1 parent 91d4388623
commit 391bd7bdcf
1 file changed
+7
+7
View File
@@ -62,6 +62,9 @@ FUNCMATH fixed_t FixedHypot(fixed_t x, fixed_t y)
angle_t angle;
fixed_t dist;
I_Assert(x != INT32_MIN);
I_Assert(y != INT32_MIN);
x = abs(x);
y = abs(y);
@@ -77,6 +80,10 @@ FUNCMATH fixed_t FixedHypot(fixed_t x, fixed_t y)
if (!y)
return x;
// prevent underflow
if (y == INT32_MIN)
return INT32_MAX;
angle = (tantoangle[FixedDiv(y, x)>>DBITS] + ANGLE_90) >> ANGLETOFINESHIFT;
// use as cosine