Skip to content

[fpmp] Shorter fp32mp2 math kernels at the low and default levels, mostly #12641's at high - #12646

Open
rmlarsen wants to merge 60 commits into
NVIDIA:mainfrom
rmlarsen:fpmp-level-kernels
Open

rmlarsen wants to merge 60 commits into
NVIDIA:mainfrom
rmlarsen:fpmp-level-kernels

Conversation

@rmlarsen

@rmlarsen rmlarsen commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Stacked on #12641 and on #12568 below it. For six fp32mp2 function families this adds a second, shorter kernel and selects between the two by accuracy level, the way #12641 already does for fmod:

Family fp32mp2_low and fp32mp2 (default) fp32mp2_high
erfc #12641's series below 1/2, one reciprocal and a degree-18 polynomial above #12641's kernel
erfcx, normcdf the one-reciprocal kernel of erfc #12641's kernel
sin, cos, tan, sincos, sinpi, cospi, sincospi one-fma fast path, minimax kernels #12641's kernels
asin, acos, atan, atan2 one table-driven kernel #12641's polynomial kernels
exp, exp2, exp10, log, log2, log10 on the tables of pow, at every level same
expm1, log1p on the tables of pow the same, plus a Taylor series for |x| < 2⁻⁷
sinh, tanh, atanh on the exp and log kernels #12641's formulas on those kernels
cosh, asinh, acosh on the exp and log kernels, at every level same
sqrt, rsqrt #12641's kernels a second-order kernel
norm3d, norm4d, rnorm3d, rnorm4d a pair kernel, no FP64 #12641's double kernels
rhypot the pair kernel, at every level same

The default level gets the faster kernel and high the more accurate one; for sqrt and rsqrt that means #12641's kernel stays the default. Where the high kernel is #12641's, its results are #12641's bit for bit on every range tested; erfcinv and normcdfinv are not in the table but call log, and change with it at every level (see Accuracy).

One commit per family, each with its own lit test at all three levels (erfcx and normcdf have a commit and a test of their own).

The kernels added here carry the edge-case fixes of #12568 where they apply: erfcx at every level is finite up to the argument where 2·exp(x²) overflows (x = −0x1.2c3cbcp+3, decided by its low limb) and returns a subnormal number instead of 0 for x ≥ 2¹²⁷ on the device, and the default-level sin returns a zero argument with its sign only when both limbs are zero. The fp64mp2 sinh, tanh, asinh, atanh, expm1 and log1p return a zero argument unchanged only when both limbs are zero, as #12568 does for atan, asin and atan2. atan2 reads an operand with a zero high limb over a nonzero low limb as its low limb, as in #12568, at every level: the entry code is shared by the two kernels.

The fp32mp2 kernels of expm1, log1p, sinh, tanh, asinh and atanh take the size of their argument, and with it the result f(x) = x of a tiny one, from the high limb. At the low level an argument whose low limb is above 2⁻²⁰ of its high limb is therefore renormalized before the kernel: a low-accuracy pair with a zero high limb over a large low limb, such as (2²⁴, 0.25) − 2²⁴ = (0, 0.25), gets the result of 0.25 (expm1: 0.284), as in #12568.

Throughput

Speedup over #12641 at 8989bc7 with the contraction fix (the code of its head, c330b4c) and the cbrt seed of #12568, that is, before that PR's special-value fixes; their own cost is listed there and is included in the high rows below. Device: evaluations per second of a kernel that calls the function 2048 times per thread on 2²⁰ arguments, best of 7 launches. Host: ns per call on one core, 2048 arguments, fastest of 40 rounds (M4: 300 rounds, minimum of six alternating passes). Every measurement ran alone on an otherwise idle machine, base and branch alternating, in two passes that agree within 3% unless a range is given.

  • A: NVIDIA RTX PRO 6000 Blackwell (sm_120) and AMD Ryzen 7 7800X3D, CUDA 13.3, GCC 15.2 -O3 -march=x86-64-v3
  • B: NVIDIA RTX 6000 Ada Generation (sm_89) and AMD Ryzen Threadripper PRO 7965WX, CUDA 13.4, GCC 15.2, same flags
  • M4: Apple M4, Apple clang 21 -O3 -mcpu=apple-m4

Default level

Function, range GPU A GPU B x86 A x86 B M4
erfc [0, 1/8] 0.99× 1.00× 0.99× 0.90–0.99× 1.01×
erfc [-4, 4] 1.20× 1.12–1.25× 1.68× 1.68× 1.79×
erfc [4, 8.2] 1.28× 1.25–1.36× 1.72× 1.72× 1.83×
sin / cos / tan [-0.78, 0.78] 1.62 / 1.60 / 1.26× 1.54 / 1.57 / 1.20× 1.47 / 1.58 / 1.52× 1.49 / 1.51–1.59 / 1.39× 1.28 / 2.36 / 1.13×
sin / cos / tan [-100, 100] 1.52 / 1.50 / 1.26× 1.45 / 1.46 / 1.23× 1.77 / 1.73 / 1.52× 1.61–1.79 / 1.76 / 1.53× 1.49 / 2.22 / 1.38×
sin [2·10⁶, 10³⁰] 1.12× 1.23× 1.33× 1.34× 1.09×
sinpi / cospi [-2, 2] 1.13 / 1.12× 1.12 / 1.13× 1.16 / 1.17× 1.17 / 1.18× 1.15 / 1.17×
asin / acos [-1, 1] 3.43 / 3.74× 3.18–3.45 / 3.26× 2.14 / 2.41× 2.15 / 2.16× 1.89 / 1.92×
atan [-1, 1] / [1, 1000] 2.15 / 2.14× 2.19 / 2.20× 2.83 / 2.93× 2.58 / 2.62–2.92× 3.35 / 3.19×
atan2 [-10, 10] 2.24× 2.24× 2.86× 2.88× 2.45×
exp / exp2 / exp10 [-10, 10] 2.29 / 2.28 / 2.04× 1.73 / 1.84 / 1.66× 3.06 / 3.80 / 2.96× 3.11 / 3.49 / 2.70× 2.40 / 2.77 / 2.32×
expm1 [-1, 1] / [-10, 10] 2.95 / 2.86× 2.40 / 2.20× 1.89 / 2.05× 1.72 / 2.08× 1.75 / 1.85×
log / log2 / log10 [10⁻¹⁰, 10¹⁰] 1.28 / 1.24 / 1.19× 1.29 / 1.29 / 1.26× 2.97 / 2.48 / 2.32× 2.73 / 2.25 / 2.11× 2.66 / 2.57 / 2.41×
log1p [-0.5, 1] / [10⁻¹⁰, 10¹⁰] 1.42 / 1.42× 1.67 / 1.73× 2.38 / 2.07× 2.16 / 1.90–2.11× 2.30 / 2.00×
sinh [-1, 1] / [-20, 20] 0.87 / 2.28× 0.88 / 1.76× 1.48 / 2.28× 1.51 / 2.11–2.35× 1.18 / 1.83×
cosh [-20, 20], tanh [-5, 5] 2.14, 1.71× 1.60, 1.68× 2.42, 3.08× 2.22, 2.82–3.13× 2.29, 2.51×
asinh / acosh / atanh 1.29 / 1.21 / 1.35× 1.37 / 1.33 / 1.31× 2.33 / 1.73 / 1.99× 2.34 / 1.74 / 1.86–2.01× 2.78 / 2.0 / 2.07×
sqrt, rsqrt, hypot 1.00–1.02× 0.99–1.00× 0.97–0.99× 1.00× 1.00×
rhypot 1.41× 1.05× 1.20× 1.20× 1.12×
norm3d / rnorm3d / norm4d / rnorm4d 43 / 41 / 51 / 48× 19 / 19 / 19 / 19× 1.32 / 1.41 / 1.35 / 1.42× 1.30 / 1.24–1.38 / 1.21–1.33 / 1.25× not measured

fp32mp2_high

Function GPU A GPU B x86 A x86 B M4
erfc, trig, sinpi, cospi 0.96–1.05× 0.93–1.01× 0.98–1.09× 0.88–1.15× 0.99–1.01×
asin, acos, atan 0.95–1.00× 0.93–1.01× 0.96–1.00× 0.97–1.00× 0.99–1.02×
atan2 0.91× 0.94× 1.02× 1.02× 0.90×
exp, exp2, exp10, log, log2, log10 as at the default level
expm1 [-1, 1] / [-10, 10] 2.28 / 2.48× 2.00 / 2.10× 1.89 / 2.04× 1.72 / 1.87× 1.76 / 1.85×
log1p [-0.5, 1] / [10⁻¹⁰, 10¹⁰] 1.32 / 1.21× 1.44 / 1.30× 2.36 / 2.27× 2.16–2.41 / 2.08× 2.29 / 2.14×
sinh [-1, 1] / [-20, 20] 1.00 / 1.42× 1.03 / 1.29× 0.98 / 1.98× 0.99 / 1.78× 1.01 / 1.77×
tanh, atanh 1.25, 1.27× 1.26–1.32, 1.27× 1.81, 1.48× 1.82, 1.49× 1.60, 1.64×
sqrt / rsqrt 0.43 / 0.52× 0.48 / 0.58× 1.03 / 0.93× 1.03 / 0.93× 0.92 / 0.94×
norms 1.00× 1.00× 0.78–0.86× 0.75–0.93× not measured

erfcx and normcdf were measured separately and differently. Host: ns per call on an 11th Gen Intel Core i5-11500T, GCC 15.2 -O3 -march=x86-64-v3, one core of an otherwise idle machine, median of three alternating passes. Device: executed instructions per call (ncu, smsp__inst_executed) of a kernel on sm_89 whose 2²⁰ arguments mix the cases of the function, not time.

Function, range x86, default x86, high Device instructions, default Device instructions, high
erfcx [-3, 10] 102.9 → 67.0 ns (1.54×) 103.0 → 106.8 ns (0.96×) 35.2 → 19.6 (1.80× fewer) 35.2 → 21.6 (1.63× fewer)
normcdf [-5, 5] 151.2 → 109.7 ns (1.38×) 1.00× 22.5 → 20.4 (1.10× fewer) unchanged

The device counts are with #12568's erfcx fixes (0.2 to 0.3 instructions per call more); the x86 times were taken before them. erfcx executes fewer instructions at high too, with the same results: its exp and its kernel are now each written once and shared by the cases that need them, where a warp whose threads take different cases used to execute a copy per case.

Slower than #12641:

Accuracy

Maximum relative error against MPFR in u² (u = 2⁻²⁴), 2M random pairs with full low limbs per range, on the M4. For the three erfc rows the entry is maximum / mean.

Function, range #12641 default high
erfc [0, 1/8] 0.88 / 0.10 0.88 / 0.10 0.88 / 0.10
erfc [-4, 4] 4.16 / 0.28 9.53 / 0.47 4.16 / 0.28
erfc [4, 8.2] 4.40 / 0.46 10.23 / 1.06 4.40 / 0.46
sin [-0.78, 0.78] / [-100, 100] / [2·10⁶, 10³⁰] 1.89 / 3.16 / 3.84 1.92 / 3.61 / 5.49 1.89 / 3.16 / 3.84
cos [-0.78, 0.78] / [-100, 100] 3.26 / 3.22 4.23 / 4.55 3.26 / 3.22
tan [-0.78, 0.78] / [-100, 100] 5.36 / 6.01 5.99 / 6.51 5.36 / 6.01
sinpi / cospi [-2, 2] 3.23 / 3.18 4.42 / 3.92 3.23 / 3.18
asin / acos [-1, 1] 2.38 / 3.33 5.71 / 5.72 2.38 / 3.33
atan [-1, 1] / [1, 1000] 1.80 / 1.71 3.85 / 3.92 1.80 / 1.71
atan2 [-10, 10] 3.18 5.40 3.18
exp / exp2 / exp10 [-10, 10] 3.23 / 2.55 / 2.96 2.85 / 2.72 / 2.93 same as default
expm1 [-1, 1] / [-10, 10] 2.76 / 3.24 3.91 / 3.51 3.91 / 3.51
expm1 [-2⁻⁷, 2⁻⁷] 1.88 4.04 1.88
log [0.5, 2] / [10⁻¹⁰, 10¹⁰] 6.76 / 5.30 4.31 / 4.70 same as default
log2 / log10 [10⁻¹⁰, 10¹⁰] 5.73 / 7.43 3.94 / 4.05 same as default
log1p [-0.5, 1] / [10⁻¹⁰, 10¹⁰] 6.28 / 5.34 5.70 / 6.31 5.70 / 5.56
log1p [-2⁻⁷, 2⁻⁷] / [2⁻⁷, 2⁻⁴] 1.79 / 2.06 8.86 / 7.76 1.79 / 7.76
sinh [-1, 1] / [-20, 20] 2.69 / 3.39 4.42 / 5.31 2.69 / 3.62
cosh [-20, 20] 3.11 3.38 3.38
tanh [-5, 5] 2.21 6.53 2.21
asinh [-100, 100] / acosh [1, 100] 4.83 / 5.02 4.91 / 4.63 same as default
atanh [-1, 1] 2.74 7.93 5.30
sqrt / rsqrt [10⁻¹⁰, 10¹⁰] 2.41 / 7.71 2.41 / 7.71 0.50 / 2.59
hypot / rhypot [10⁻⁵, 10⁵] 2.83 / 7.71 2.83 / 7.71 2.83 / 7.71
norm3d / norm4d [10⁻⁵, 10⁵] 0.55 / 0.58 5.04 / 4.33 0.55 / 0.58
rnorm3d / rnorm4d [10⁻⁵, 10⁵] 0.55 / 0.56 7.65 / 8.09 0.55 / 0.56

Where the default level is less accurate than #12641, high restores #12641's error, except:

erfcx and normcdf, 1M pairs per range on x86-64: erfcx is 7.3 u² on [-3, 10] and 5.3 u² on [8, 2⁶⁰] at the default level (#12641 and high: 3.6 and 3.9 u²); normcdf is 6.7 u² on [-13, 0] and 7.0 u² on [-1, 1] (#12641 and high: 4.5 and 3.8 u²). On the device the largest errors are 6.1 u² (erfcx) and 7.6 u² (normcdf). The same run has 11.5 u² for erfc at x ≈ 5.55, above the 10.23 of the table's 2M pairs on [4, 8.2].

At the default level the host and an sm_89 GPU give different last bits for part of the arguments of asin and acos (1–14% per range), erfc, erfcx and normcdf (up to 10%), atanh (up to 5%) and tanh (up to 2%): their kernels start from the device's approximate reciprocal and reciprocal square root. The largest error is the same on both. #12641 has the same for rsqrt, cosh, asinh and boys_f0.

exp, log, log2 and log10 are at least as accurate as #12641 in maximum error, at every level.

Instructions

fp32 SASS instructions of a kernel that calls the function once, #12641 → default / high, CUDA 13.4:

Function sm_75 sm_89 sm_120
erfc 680 → 576 / 680 688 → 584 / 688 688 → 584 / 688
sin, cos 728 → 664 / 728 720 → 656 / 728 760 → 696 / 760
tan 736 → 704 / 744 736 → 696 / 744 776 → 736 / 776
sinpi / cospi 368 → 344 / 368, 312 → 288 / 312 368 → 344 / 368, 312 → 288 / 312 376 → 352 / 376, 328 → 304 / 328
asin / acos 456 → 144 / 472 464 → 152 / 480 464 → 152 / 480, 480 → 152 / 496
atan / atan2 248 → 128 / 264, 320 → 160 / 320 256 → 136 / 272, 328 → 168 / 328 264 → 136 / 272, 328 → 168 / 336
exp / exp2 / exp10 152 → 96, 136 → 88, 152 → 104 160 → 104, 144 → 96, 160 → 112 160 → 112, 144 → 96, 160 → 112
expm1 / log1p 288 → 208 / 256, 288 → 296 / 352 288 → 216 / 264, 296 → 304 / 360 288 → 216 / 272, 296 → 288 / 344
log / log2 / log10 152 → 128, 152 → 128, 160 → 136 160 → 136, 160 → 136, 168 → 144 168 → 144, 168 → 152, 168 → 160
sinh / cosh / tanh 304 → 160 / 240, 192 → 120, 336 → 240 / 288 312 → 168 / 248, 200 → 128, 344 → 248 / 296 304 → 176 / 256, 192 → 128, 344 → 248 / 296
asinh / acosh / atanh 512 → 552, 504 → 568, 472 → 544 / 800 520 → 552, 512 → 576, 480 → 544 / 816 528 → 552, 520 → 568, 488 → 536 / 784
sqrt / rsqrt 32 → 32 / 88, 32 → 32 / 80 40 → 40 / 96, 40 → 40 / 88 48 → 48 / 96, 48 → 48 / 88

#12641's counts are the same at both levels. asinh, acosh, atanh and the default log1p have more instructions than #12641 and are still faster.

Testing

  • New lit tests, host and device, each at fp32mp2, fp32mp2_low and fp32mp2_high with a bound that fits the level: math_erfc, math_erfcx (erfcx and normcdf), math_trig, math_invtrig, math_explog, math_hyperbolic, math_sqrt.
  • Host. All fpmp/math* tests, accuracy and scalar_ops pass on the M4 at -O2, -O0 and -O2 -ffp-contract=off, at every commit of the branch at -O2, up to the sqrt commit; the erfcx and normcdf commit and the newer commits of [fpmp] fp32mp2 math on top of #12641: contraction-independent host results, special-value fixes, faster host cbrt seed #12568 were run on x86-64 only (lit, below).
  • lit on the GPU. The full cuda/fp/units suite passes on sm_89: 68 passed, 2 unsupported (fpemu/float64, fpmp/atomic_dd), CUDA 13.4 and GCC 15.2.
  • --use_fast_math (the CCCL_ENABLE_FASTMATH job of the PR matrix). The suite passes on sm_89 with it as well, 68 passed and 2 unsupported, and the fpmp/math* tests pass with it on sm_75 (GTX 1650 and RTX 2070). Fast math flushes subnormals to zero on the device, so every table row with a subnormal operand, or with a result below 2⁻⁷⁸, whose low limb can be subnormal, is left to the IEEE build.
  • Compile modes. Header mode and _CCCL_FPMP_USE_LIB are clean at C++17 and C++20 for fp32mp2, low, high, fp64mp2 and fp64mp2_high.

Not run: MSVC, sm_75 and sm_120 lit, data-center GPUs. fp64mp2 is covered by the unit tests only; its high entry points forward to the single fp64mp2 kernel, except sqrt, rsqrt, which get the second-order kernel, and the norms, which keep the pair kernel (a double fallback would halve their precision).

Open question for #12641

fp32mp2_high selecting a different math kernel is #12641's own mechanism for fmod, remainder and remquo. This PR uses it for 22 more functions. If the levels are meant to govern arithmetic only, the alternative is to keep one kernel per function, and then each family needs a choice between the two columns above.

Merge order

#12641, then #12568, then this. It carries the stack's CI override, which is reset before merging.

Checklist

  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

🤖 Generated with Claude Code

akolesov-nvidia and others added 30 commits November 13, 2026 21:35
exp: 41 -> 46 bits on the working range, 43/41 -> 46 on the normal and
special datasets, same throughput, latency 328 -> 308 clk/ev. Results
below FLT_MIN now underflow to subnormals instead of zero.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
exp2: 41 -> 46 bits. exp10: 44 -> 46 bits, +37 % throughput.
log, log1p, log2, log10: 43-44 -> 45 bits near 1, 45 -> 46-48
elsewhere, 10-26 % faster.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Bits on the work/normal/special/pattern datasets (L4):
sin 37/41/41/25 -> 46/46/46/45, cos 38/41/41/28 -> 46/46/46/45,
tan 37/41/41/25 -> 45/45/45/44, sinpi 41/41/41/23 -> 46/46/46/25,
cospi 41/41/41/41 -> 46/46/46/45; throughput +8-26 %. The exp/log
kernels give bit-identical results at 10-23 % more throughput.

Fixes: atan2 returned 0 for operands above 2^126, atan(+-inf) returned
NaN instead of +-pi/2.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
asin, acos: 44 -> 46 bits. atan: 45 -> 47 bits, +17 % throughput.
atan2: 45 -> 46 bits, +11 %. sqrt: fp32mp2 9.5 -> 2.8 u^2 (44 -> 46
bits), now the same on host and device. The error-bounds page is
updated.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
fp64mp2 sqrt returns to its original form (106 bits, 25 % faster than
the previous commit's). fp32mp2 rsqrt: 44 -> 47 bits (13.4 -> 2.0 u^2),
the same on host and device, at 2.12 instead of 4.97 ev/clk/SM on the
L4. The error-bounds page is updated.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
No functional change: exp, expm1, sinh, cosh, tanh and the log family
give identical SASS, accuracy and throughput.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
pow: normal dataset 41 -> 44 bits, special 39 -> 41, work 30 -> 33,
pattern 27 -> 32 (the last three limited by results below 2^-102).
Throughput 0.30 -> 0.25 ev/clk/SM, accepted for the accuracy gain.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
cbrt: 45 -> 47 bits, +17 % throughput. rcbrt: 46 -> 47 bits, +58 %
throughput. hypot, rhypot: native fp32mp2 implementations instead of
the double fallback, 46 bits.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
sinh: 45-46 -> 46 bits (4.4 -> 3.6 u^2). tanh: 45 -> 46 bits (6.9 ->
3.1 u^2). atanh: 44-45 -> 46 bits (8.6 -> 3.6 u^2), throughput 0.53 ->
0.44 ev/clk/SM.

Fixes: sinh and cosh returned NaN above |x| = 88.72, now +-inf; atanh
returned NaN for arguments just inside +-1.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…air; round fix

fdim, remquo and the four integer roundings no longer go through
double: exact on the limb pair (remquo over remainder's reduction, the
roundings over rint / round). The predicates, copysign, frexp, modf,
ilogb and logb test the float limbs instead of widening to double.

Fix: round was off by one on integer-valued pairs with |lo| >= 2^23;
now 0 error on the ts, 190 -> 166 clk/ev.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
isfinite, isinf, isnan and signbit classify both limbs and combine the
answers by logic alone, so a loose pair with an infinity or a NaN in lo,
or (+inf, -inf), is reported correctly; no arithmetic involved. The
function list no longer labels them as placeholders.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…mcdf, erfinv, erfcinv

erfc: 40-41 -> 45 bits (136 -> 4.8 u^2). boys_f0: 43 -> 46 bits, +27 %
throughput. normcdfinv: 41 -> 45 bits at the old size and throughput.
erfcx, normcdf, erfinv, erfcinv: native fp32mp2 implementations instead
of the device-only double fallbacks, 45-46 bits, 1.3-3x lower latency.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
normcdfinv(1 - 2^-53), (1 - 2^-40) and 2^-40 against CUDA's double
normcdfinv, and +-inf at 1 and 0.
… fpmp2_accuracy::high

The remainder family of the former dev/akolesov/fp32mp2_remainder_fix line, replayed
as one commit on the rebase onto main (its three commits built on an older base of
the math line and no longer apply one by one):

- fpmp: fp32mp2 fmod exact on sparse pairs; remainder exact in high (3ee7818)
- fpmp: fp32mp2 fmod exact in fpmp2_accuracy::high only (0652edc)
- fpmp: fp32mp2 remquo on remainder's reduction, exact in fpmp2_accuracy::high (35e2622)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Twenty expression sites in exp, the hyperbolic functions, the special functions and
asin/acos promoted a float that one limb holds exactly to a pair first (__ffloat(0.5f),
__ffloat(1.0f), __ffloat(__y0), ...) or doubled a pair as x + x. They now pass the scalar
to the mixed-type operators, or multiply by 2.0f. Today the operators promote the scalar
themselves, so values and code are unchanged (verified: identical maximum errors on the
accuracy suite); once NVIDIA/cccl PR NVIDIA#12519 lands, the same sites take its single-limb
kernels (6 operations and a 2u^2 bound for the product), measured at 1.053x throughput
over the fp32mp2 math set. Coefficient tables and pairs built from two limbs are unchanged.

(cherry picked from commit 47c0752)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…rounding functions, normcdfinv and boys_f0

After NVIDIA/cccl issue NVIDIA#12550. Wrong results for special arguments, now C99 Annex F:
- log, log2, log10 returned finite numbers for x <= 0, +inf and NaN (log(-2) = 0.693,
  log(0) = -104.7, log(+inf) = 88.7): +-0 -> -inf, +inf -> +inf, negative or NaN -> NaN.
- atan turned NaN into pi/2 in its reciprocal range: NaN propagates.
- floor, ceil, trunc, round, rint returned NaN for +-inf (inf - inf in the lo-limb accumulate):
  the infinity itself.
- normcdfinv returned +-inf outside [0, 1]: NaN; exactly 0 and 1 still give -inf and +inf.
- boys_f0 returned NaN below about -94, where F0 exceeds FLT_MAX: +inf.

All guards are branch-free: the core runs and a few selects pick the special value afterwards
(a uniform branch to a cold path cost two to three times more in these throughput-bound kernels,
its convergence barrier more than its comparisons). Measured on an RTX A6000 against the
previous tip: atan, floor, ceil, trunc, round within 1%, boys_f0 -2%, rint/nearbyint -3%,
log/log2 -3.5%, normcdfinv -4.5%, log10 -7%; bit counts unchanged on every cell. sqrt and rsqrt
are deliberately left without special-value handling (see the docs); signed-zero results of the
odd functions are unchanged.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
State what the two core-header roots do for zero, infinity, negative, NaN and (on the device)
subnormal arguments, and the scaling workaround, since they are deliberately kept free of the
special-value handling the math functions have.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
(cherry picked from commit 806124c)
Adds libcudacxx/test/libcudacxx/cuda/fp/units/fpmp/math_special_values.pass.cpp,
which checks the class and the sign of the result at the arguments of C99 Annex F:
zero, a negative argument, the largest finite values, the infinities and NaN, for
log/log2/log10, atan, the six rounding functions (fp32mp2 and fp64mp2), normcdfinv
and boys_f0. sqrt and rsqrt are left out (documented behavior), and so is the sign of
a zero result. The test runs on the host and in a kernel through force_include.h and
fails on the code before the special-value fixes (log(0) was the first assertion to
trip).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Documents, next to the sqrt/rsqrt note, that fpmp2 arithmetic and math do not track
the sign of zero: renormalization adds the two limbs, so every zero result of +, -, *
and fma is +0 even where IEEE 754 gives -0, while a zero from / or from a math
function carries whichever sign its evaluation leaves (and fp32mp2 floor(-0) differs
between host and device). Only negation, fabs, copysign and construction from a
built-in -0.0 keep the sign. Measured on host and device for fp32mp2 and fp64mp2.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…2548

Replaces the pow section of fpmp_math_impl_pow.h with the one from NVIDIA/cccl
PR NVIDIA#12548 at 6490471 (Rasmus Munk Larsen: a triple-float ln|a| and a
table-driven exp, tables with hidden visibility in host code), byte for byte, and
adds its unit test math_pow.pass.cpp. The cbrt, rcbrt, hypot and norm code of
this header stays as it is on this branch.

Measured on an RTX A6000 with the cccl-fpdev suite, fp32mp2 pow at the default
level, this implementation against the one it replaces: work 34 vs 33 bits,
normal 46 vs 44, special 46 vs 41, pattern 36 vs 32 (warnings 6e-04% vs 16.85%),
and 1.45x the throughput (0.45 vs 0.31 ev/clk/SM, 422 vs 672 clk/ev). cbrt,
rcbrt, hypot and rhypot unchanged.

When NVIDIA#12548 lands upstream, its version of the pow section is the one to keep on
the next rebase of this branch.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The new fp32mp2 rsqrt form of this branch read 47 bits instead of 44 but at 0.44x the
throughput (2.30 vs 5.28 evaluations per clock per SM on an RTX A6000), too high a price
for three bits in a core operation. __fpmp2_rsqrt goes back to the form on main; the new
sqrt form (46 bits, 1.14x) stays. The special-argument behavior documented in fpmp.rst is
the same for both forms (NaN for +inf, a negative argument, NaN and, on the device, a
subnormal; rsqrt(0) is NaN).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…A#12530

Our Cody-Waite fast path (|x| < 2^20) folded the second- and third-order terms with
error-free sums but closed with float additions whose rounding is relative to those
terms, about 2^-79 in absolute value; at an argument within 2^-40 of a multiple of
pi/2 that is 300 u^2 relative to the reduced argument (main's math_reduction test, sin
of (0x1.b751ep+16, 0x1.a9937cp-12)). NVIDIA#12530 accumulates the corrections in
fp32mp2_high, whose error is relative to the running result, and passes its test at
<= 6 u^2. This takes that block as it is on main; the Payne-Hanek slow path and the
sin/cos/tan kernels stay ours. Measured on an RTX A6000: bits unchanged (sin 46/45,
cos 46/45, tan 45/44 on normal/pattern), sin and cos 0.49 instead of 0.54 and tan 0.45
instead of 0.48 evaluations per clock per SM.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Two lines the pre-commit.ci clang-format check reformats: the nested conditional in
__internal_fpmp2_log_special_value and an assignment alignment in normcdfinv.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- erfcinv: a pair just below 2, (2, -eps), is a valid argument and gave -inf because only
  the high limb was tested against 2; -inf needs lo == 0 as well.
- exp: the float at the overflow threshold, 0x1.62e430p+6 = 88.72284 > ln(FLT_MAX), passed
  the guard, overflowed in the final product and came back as (NaN, NaN) through
  renormalize; the core maps an infinite product to (+inf, 0). expm1, sinh and cosh
  inherited the NaN. exp2 and exp10 overflowed to (inf, -inf); their scaling helper
  returns (inf, 0).
- sinh and cosh returned +-inf from 88.7228 on, while the result is finite up to 89.416:
  they take e^|x|/2 straight from exp with its power-of-two scale lowered by one binade
  (exp_impl<-1>, exact), and the second term from that half (0.25 / (e/2)); cosh no longer
  doubles it back.
- fdim: a finite difference that overflows (fdim(FLT_MAX, -FLT_MAX)) is +inf, not the NaN
  of the overflowed subtraction.
- hypot: (inf, 0) on overflow instead of (inf, -7.7e30), which did not compare equal to inf.
- modf(+-inf) = (+-0, +-inf) and NaN for NaN (C99 F.10.3.12); it returned a NaN fraction.
- ilogb returns FP_ILOGB0, FP_ILOGBNAN and INT_MAX directly, without the double detour.
- llrint and llround (and lrint, lround through them) route a non-finite low limb to the
  scalar fallback as well; the cast of the NaN the kernel produced was undefined.
- The internal trunc, floor and ceil of the limb type use the builtins on both targets:
  one FRND on the device instead of an int round trip, and they keep the sign of a zero
  result where the round trip returned +0 for every |x| < 1, which is where the host and
  the device disagreed on floor(-0) and ceil(-0.5). round keeps the sign on its zero-high-
  limb path, ceil and trunc on their +-1 correction path, so the whole rounding family
  returns a zero with the sign of the argument (C99 F.10.6) on the host and the device.
- fpmp_math_impl_lib.h no longer declares mid_/low_ fmod, remainder and remquo entry
  points: no kernel and no caller existed for them.

Measured on an RTX A6000 (fpdev timing cells, before -> after, ev/clk/SM): floor 7.00 ->
7.55, ceil 7.01 -> 7.44, trunc 7.13 -> 7.72, round 3.69 -> 4.00, rint and nearbyint 4.47 ->
4.82 (the builtins are cheaper than the int round trip); exp 0.84 -> 0.83, exp10 1.16 ->
1.14, cosh 0.66 -> 0.64, hypot 1.69 -> 1.59, erfcinv 0.41 -> 0.40; everything else within
1%. No bit count changed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The exponential family at its edges (the overflow threshold of exp and the float below it,
exp2 at 128, exp10 near 38.53, +-inf, NaN), sinh and cosh through 89.416 and past it,
tanh, asinh, acosh and atanh at their ends, asin and acos beyond |x| = 0.575 and outside
[-1, 1], the quadrants, axes and infinities of atan2, sinpi and cospi at integers and
half-integers, sin, cos and tan in the Payne-Hanek range (|x| >= 2^20), cbrt, rcbrt, hypot
(canonical overflow) and rhypot, fdim including its overflow, erf, erfc, erfcx, normcdf,
erfinv and erfcinv at their ends and the pairs just inside them, modf at +-inf and NaN, the
classification predicates on loose pairs and the integer conversions of such pairs. The
rounding family now asserts the sign of a zero result. The erfcinv bug at (2, -eps) and
the exp NaN at the threshold both trip the earlier version of this test.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…he rounding functions promise

- The radian reduction is exact (Payne-Hanek) from |x| >= 2^20; below that the four-piece
  pi/2 leaves about n * 2^-100 absolute in the reduced argument, so the near-zero one of
  sin and cos loses relative accuracy within n * 2^-54 of a multiple of pi/2 (a few u^2 at
  2^-40 from a multiple near 1e5, 300 u^2 at 2^-53 from 161 pi/2, up to 1e6 u^2 for the
  pairs closest to a multiple near 2^19), measured with a search over the fast-path range.
- sinh and cosh stay finite up to |x| = 89.416.
- sqrt keeps its accuracy down to a high limb of about 2^-100; below that the residual is
  subnormal and the error grows (11 u^2 at 2^-108).
- The exact high level of fmod, remainder and remquo needs the divisor's limbs within 126
  bits; the fp64mp2 remquo, lrint, lround, llrint and llround are double fallbacks.
- The sign of zero: the rounding functions and the fraction of modf now carry the sign of
  the argument on both targets; atan2 reads a zero argument as +0.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…he subnormal cases

- fpmp_impl_divsqrt.h: the fp32mp2 sqrt declared `__one` for one if-constexpr branch only;
  nvcc's "declared but never referenced" (177-D) is an error on the GCC 7 and MSVC jobs and
  took seven cuda/fp units down with it. The constant is spelled at its single use.
- math_special_values.pass.cpp and math_pow.pass.cpp: `--use_fast_math` flushes subnormals
  to zero on the device, so the assertions whose argument is a subnormal (log family and
  normcdfinv at denorm_min, pow with a subnormal base or a subnormal low limb) are left to
  the IEEE build, keyed on TEST_HAS_CUDA_FAST_MATH as the lit configuration defines it.
  Reproduced and verified locally with `nvcc --use_fast_math` on sm_86.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…d the mixed Horner step

Four float products in fpmp_math_impl.h were formed with a plain multiply or __fpmp_mul_rn
and then added by the caller: the low quotient of the refined division (both the exact-
remainder and the plain path) and the transition step of the mixed-precision Horner
evaluator (fast and standard variants; main had __fpmp_mul_rn_nocontract there). With GCC's
default -ffp-contract=fast the host fuses such a product into the following add, and 29 of
40 fp32mp2 math functions returned different bits from the non-contracting builds (-O0,
-ffp-contract=off, no-FMA targets, Clang) and from the device. The four sites now use
__fpmp_mul_rn_nocontract (fma(x, y, 0), which a compiler cannot fuse further); every host
build then agrees bit for bit, and with the device except where the device seeds rsqrt and
the reciprocal with its approximate instructions.

Device cost: the helper is one FFMA in place of one FMUL; the SASS instruction counts of the
14 affected cells are unchanged (tan 600 -> 608), throughput within noise (0.96-1.03x), no
bit changes. Found by rmlarsen in the review of NVIDIA#12641.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…set before merging]

This PR changes only the fpmp headers and their tests, so limit its CI to the cuda/fp
lit suite on a T4 (gcc and clang, CTK 12.X and 13.X) and an H100 (gcc, CTK 13.X), plus
an MSVC precompile of the same suite. The override must be reset to empty before the
PR can merge.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rmlarsen rmlarsen changed the title [fpmp] Shorter fp32mp2 math kernels at the low and default levels, #12641's at high [fpmp] Shorter fp32mp2 math kernels at the low and default levels, mostly #12641's at high Nov 15, 2026
@rmlarsen
rmlarsen marked this pull request as ready for review November 15, 2026 02:37
@rmlarsen
rmlarsen requested review from a team as code owners November 15, 2026 02:37
@cccl-authenticator-app cccl-authenticator-app Bot moved this from In Progress to In Review in CCCL Nov 15, 2026
@coderabbitai

coderabbitai Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • Resume reviews
  • Trigger review

No actionable comments were generated in the recent review.

Summary

Summary by CodeRabbit

  • New Features
    • Added high-accuracy options for more multiprecision math functions, including roots, exponentials, logarithms, trigonometric and hyperbolic functions, probability functions, norms, and remainder operations.
  • Bug Fixes
    • Improved handling of signed zero, infinities, NaNs, subnormals, and values near domain and range boundaries.
    • Corrected rounding, classification, and special-value results for edge cases.
  • Documentation
    • Expanded accuracy guidance and documented special-value behavior across accuracy levels.
  • Tests
    • Added host and device coverage for multiprecision math accuracy and special values.

FPMP math implementations gain revised kernels and high-accuracy dispatch across several function families. The change adds pair-aware special-value and rounding behavior, expands host/device accuracy and boundary tests, updates FPMP documentation, and adds targeted CI configurations.

Changes: FPMP math implementation

Layer / File(s) Summary
Square-root kernels and shared numerical helpers
libcudacxx/include/cuda/__fp/fpmp_impl.h, libcudacxx/include/cuda/__fp/fpmp_impl_divsqrt.h, libcudacxx/include/cuda/__fp/fpmp_math_impl.h, libcudacxx/include/cuda/__fp/fpmp.h, docs/libcudacxx/extended_api/fp/fpmp_error_bounds.rst, libcudacxx/test/libcudacxx/cuda/fp/units/fpmp/math_sqrt.pass.cpp
Adds high-accuracy sqrt and rsqrt kernels and dispatch. Adds refined pair division and a fast mixed-Horner strategy. Adds reference-accuracy and special-value tests, and updates square-root error-bound documentation.
Exponential and logarithmic kernels
libcudacxx/include/cuda/__fp/fpmp_math_impl_exp.h, libcudacxx/include/cuda/__fp/fpmp_math.h, libcudacxx/test/libcudacxx/cuda/fp/units/fpmp/math_explog.pass.cpp
Replaces fp32mp2 exponential and logarithmic implementations with table-driven kernels. Adds high-accuracy small-input paths and tests for precision, ranges, and special values.
Trigonometric and inverse-trigonometric kernels
libcudacxx/include/cuda/__fp/fpmp_math_impl_trig.h, libcudacxx/include/cuda/__fp/fpmp_math.h, libcudacxx/test/libcudacxx/cuda/fp/units/fpmp/math_trig.pass.cpp, libcudacxx/test/libcudacxx/cuda/fp/units/fpmp/math_invtrig.pass.cpp
Expands Payne–Hanek reduction to 96 bits and adds separate minimax and high-accuracy kernels. Updates inverse-trigonometric paths and tests for accuracy and special values.
Hyperbolic kernels
libcudacxx/include/cuda/__fp/fpmp_math_impl_hyperbolic.h, libcudacxx/include/cuda/__fp/fpmp_math.h, libcudacxx/test/libcudacxx/cuda/fp/units/fpmp/math_hyperbolic.pass.cpp
Revises hyperbolic and inverse-hyperbolic kernels, including selected high-accuracy paths and limb-aware domain handling. Adds accuracy and boundary tests.
Power, roots, and norms
libcudacxx/include/cuda/__fp/fpmp_math_impl_pow.h, libcudacxx/include/cuda/__fp/fpmp_math.h, libcudacxx/test/libcudacxx/cuda/fp/units/fpmp/math_pow.pass.cpp, libcudacxx/test/libcudacxx/cuda/fp/units/fpmp/math_cbrt.pass.cpp, libcudacxx/test/libcudacxx/cuda/fp/units/fpmp/math_sqrt.pass.cpp
Revises pow, cbrt, rcbrt, and scaled norm implementations. Adds tests for reference accuracy, exact results, and exceptional inputs.
Error and probability functions
libcudacxx/include/cuda/__fp/fpmp_math_impl_special.h, libcudacxx/include/cuda/__fp/fpmp_math.h, libcudacxx/test/libcudacxx/cuda/fp/units/fpmp/math_erfc.pass.cpp, libcudacxx/test/libcudacxx/cuda/fp/units/fpmp/math_erfcx.pass.cpp
Revises error and probability kernels, including endpoint and tail handling. Adds tests for error bounds, subnormal results, saturation, and probability endpoints.
Remainder, rounding, and pair classification
libcudacxx/include/cuda/__fp/fpmp_math_impl_nearint.h, libcudacxx/include/cuda/__fp/fpmp_math_impl_classify.h, libcudacxx/include/cuda/__fp/fpmp_math_impl_manip.h, libcudacxx/include/cuda/__fp/fpmp_math_impl_lib.h, libcudacxx/test/libcudacxx/cuda/fp/units/fpmp/math.pass.cpp, libcudacxx/test/libcudacxx/cuda/fp/units/fpmp/math_special_values.pass.cpp
Adds exact sparse-pair remainder paths and updates rounding, manipulation, and classification behavior. Adds tests for special values, sparse pairs, and limb-aware results.
FPMP docs and CI coverage
docs/libcudacxx/extended_api/fp/fpmp.rst, docs/libcudacxx/extended_api/fp/fpmp_spec.rst, ci/matrix.yaml
Updates operation and special-value documentation. Adds targeted GPU and CPU-precompile configurations for FPMP tests.

Change: Refactor

Merge Risk: Moderate · previous review 62496

The CI override must be reset before merging, or the required checks will not pass. A few pair edge cases, such as NaN low limbs and opposite-infinity pairs, behave inconsistently with the new classification predicates. The author has accepted this as a design choice. The fast-math test failure in the erfcx test has been fixed.

Security architecture risk: Minimal · up to 62496

The inspected changes alter numerical evaluation and accuracy selection without establishing a new security boundary, elevated authority, or material failure-containment risk. No introduced or worsened security-relevant architecture concern was identified.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • inferred — The demonstrated exposure is to callers receiving changed numerical results from existing FPMP interfaces. No inspected path connects those results to a privileged sink; security-sensitive downstream use remains outside the inspected coverage.
- [ ] Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Recent review info

Run configuration

  • Source mode: Auto
  • Repository: NVIDIA/cccl
  • Branch: main
  • File: .coderabbit.yaml
  • Commit: 8d33eb1
  • Layers: Repository YAML
  • Review profile: CHILL
  • Plan: Enterprise
  • Run ID: 2546f780-81ec-46a9-bbc9-277a2745e0aa
Commits

Reviewing files that changed from the base of the PR and between 5a9b39b and 62496ed.

Files selected for processing: 8

  • libcudacxx/include/cuda/__fp/fpmp_math_impl_classify.h
  • libcudacxx/test/libcudacxx/cuda/fp/units/fpmp/math_erfc.pass.cpp
  • libcudacxx/test/libcudacxx/cuda/fp/units/fpmp/math_erfcx.pass.cpp
  • libcudacxx/test/libcudacxx/cuda/fp/units/fpmp/math_explog.pass.cpp
  • libcudacxx/test/libcudacxx/cuda/fp/units/fpmp/math_hyperbolic.pass.cpp
  • libcudacxx/test/libcudacxx/cuda/fp/units/fpmp/math_invtrig.pass.cpp
  • libcudacxx/test/libcudacxx/cuda/fp/units/fpmp/math_special_values.pass.cpp
  • libcudacxx/test/libcudacxx/cuda/fp/units/fpmp/math_sqrt.pass.cpp

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 8 remain after this review.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 16

Nitpick comments: 3

libcudacxx/test/libcudacxx/cuda/fp/units/fpmp/math_cbrt.pass.cpp: 1

147-148: Trivial · Maintainability & Code Quality · Quick win

suggestion: Tighten the 16u^2 bound so the test detects a loss of the new accuracy.

The new kernels document 1.5u^2 (cbrt) and 1.4u^2 (rcbrt) worst case against binary128. The old device Halley step is documented at 7u^2. A regression to the old error, or worse, still passes <= 16.0. This test is the only accuracy guard for the rewritten Halley step. Use a bound of about 4.0, then check it on host and device. Keep the margin for the seed-independent claim.

libcudacxx/test/libcudacxx/cuda/fp/units/fpmp/math_pow.pass.cpp: 1

114-177: Trivial · Maintainability & Code Quality · Quick win

suggestion: Add main-path cases for a = +1 and a = -1 with finite exponents.

The kernel's special-value block is skipped for a finite positive a and finite nonzero b. This includes a == 1, and a == -1 with an integer b. Both go through __internal_fpmp2_pow_log, __internal_fpmp2_pow_exp and __internal_fpmp2_exp_scale. C99 requires exactly 1 for pow(+1, b) and -1 or +1 for pow(-1, b) by parity. No test covers pow(+1, 2.5) or pow(-1, 3). Add cases such as {1, 0, 2.5, 0, 1}, {1, 0, -1e10, 0, 1} and {-1, 0, 3, 0, -1}. check_exact then also checks that lo is zero.

libcudacxx/test/libcudacxx/cuda/fp/units/fpmp/math_sqrt.pass.cpp: 1

116-117: Trivial · Maintainability & Code Quality · Quick win

suggestion: Tighten the tolerance_high entries so the high level is distinguishable from the short kernel.

The header comment documents the high-level sqrt at 0.5u^2 and rsqrt at 3.9u^2. tolerance_high[fn_sqrt] is 2.5, which equals the short kernel's documented bound. If the high-level dispatch fell back to the short sqrt kernel, the assertion would still pass. The fdim bound of 3.5 and the default-level sqrt bound of 3.5 are also above the documented 2.5.

Use bounds near the documented per-kernel errors, with a small margin. Allow the documented device and host differences for fp64mp2.

Review info

Run configuration

  • Source mode: Auto
  • Repository: NVIDIA/cccl
  • Branch: main
  • File: .coderabbit.yaml
  • Commit: 8d33eb1
  • Layers: Repository YAML
  • Review profile: CHILL
  • Plan: Enterprise
  • Run ID: ca6f9082-6d84-49c5-991f-541d14789200
Commits

Reviewing files that changed from the base of the PR and between 8d33eb1 and 0013716.

Files selected for processing: 28

  • ci/matrix.yaml
  • docs/libcudacxx/extended_api/fp/fpmp.rst
  • docs/libcudacxx/extended_api/fp/fpmp_error_bounds.rst
  • docs/libcudacxx/extended_api/fp/fpmp_spec.rst
  • libcudacxx/include/cuda/__fp/fpmp.h
  • libcudacxx/include/cuda/__fp/fpmp_impl.h
  • libcudacxx/include/cuda/__fp/fpmp_impl_divsqrt.h
  • libcudacxx/include/cuda/__fp/fpmp_math.h
  • libcudacxx/include/cuda/__fp/fpmp_math_impl.h
  • libcudacxx/include/cuda/__fp/fpmp_math_impl_classify.h
  • libcudacxx/include/cuda/__fp/fpmp_math_impl_exp.h
  • libcudacxx/include/cuda/__fp/fpmp_math_impl_hyperbolic.h
  • libcudacxx/include/cuda/__fp/fpmp_math_impl_lib.h
  • libcudacxx/include/cuda/__fp/fpmp_math_impl_manip.h
  • libcudacxx/include/cuda/__fp/fpmp_math_impl_nearint.h
  • libcudacxx/include/cuda/__fp/fpmp_math_impl_pow.h
  • libcudacxx/include/cuda/__fp/fpmp_math_impl_special.h
  • libcudacxx/include/cuda/__fp/fpmp_math_impl_trig.h
  • libcudacxx/test/libcudacxx/cuda/fp/units/fpmp/math.pass.cpp
  • libcudacxx/test/libcudacxx/cuda/fp/units/fpmp/math_cbrt.pass.cpp
  • libcudacxx/test/libcudacxx/cuda/fp/units/fpmp/math_erfc.pass.cpp
  • libcudacxx/test/libcudacxx/cuda/fp/units/fpmp/math_explog.pass.cpp
  • libcudacxx/test/libcudacxx/cuda/fp/units/fpmp/math_hyperbolic.pass.cpp
  • libcudacxx/test/libcudacxx/cuda/fp/units/fpmp/math_invtrig.pass.cpp
  • libcudacxx/test/libcudacxx/cuda/fp/units/fpmp/math_pow.pass.cpp
  • libcudacxx/test/libcudacxx/cuda/fp/units/fpmp/math_special_values.pass.cpp
  • libcudacxx/test/libcudacxx/cuda/fp/units/fpmp/math_sqrt.pass.cpp
  • libcudacxx/test/libcudacxx/cuda/fp/units/fpmp/math_trig.pass.cpp

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 10 remain after this review.

Comment thread ci/matrix.yaml
Comment on lines +92 to +93
- {jobs: ['run_gpu'], project: 'target', ctk: '12.X', cxx: ['gcc14', 'clang19'], gpu: 't4',
args: '--preset libcudacxx --lit-tests "cuda/fp/units"'}

@coderabbitai coderabbitai Bot Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Major · Functional Correctness · Quick win

important: Clear the temporary override before merging.

These entries replace the normal pull_request matrix. Lines 67-69 explicitly state that this mode makes branch-protection checks fail. Remove the temporary entries after targeted validation, but retain the override: key.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes: the override is there while this stack is in draft iteration and is reset to an empty override: before merging.

rmlarsen and others added 5 commits November 14, 2026 19:04
NVIDIA#12641's head now carries the contraction fix for the products of
div_refined and the Horner transition step, the same four changes as this
branch's own commit; the conflict is resolved with NVIDIA#12641's lines.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The seed of erfcinv(q) for q < 1.1e-7, which erfinv also uses next to +-1,
is the Abramowitz-Stegun rational 26.2.23, written with * and +. The compiler
then decides which of its products to contract: GCC's default
-ffp-contract=fast fuses them, -ffp-contract=off and Clang do not, and the
device fuses on its own terms. The seeds differ in their last bits, and after
the Halley and Newton steps so does the low limb of the result, for about 60
of 100,000 arguments.

Both polynomials are now Horner forms in __fpmp_fma_rn, the same operations in
every build. With that, result hashes over 107 ranges (37 functions, 100,000
arguments each) agree between g++ -O3, g++ -O3 -ffp-contract=off and Clang,
except sqrt for arguments below 2^-100, which is outside the range the
documentation gives it. Accuracy against MPFR is unchanged (erfinv 2.5 u^2,
erfcinv 2.6 u^2 worst), and so is the time per call.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Brings in the merge of NVIDIA#12641 (c330b4c), the erfcinv seed in
explicit fmas and the sign of zero results of sinpi, cospi, erf and erfinv.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…ults

- sinpi(-0) and the sine of sincospi(-0) returned +0: the sum of the products
  pi * x is +0 for a zero of either sign. They now return the argument.
- cospi returned -0 at the half-integers n + 1/2 with n odd after rounding
  (0.5, -0.5, 2.5, ...): the cosine there is the kernel's sine of a zero
  fraction, +0, negated for an odd n. IEEE 754-2019 has +0 at every
  half-integer. The high limb is now negated as 0 - c, which leaves +0 as it
  is and costs nothing.
- erf(-0) and erfinv(-0) returned +0: both renormalize the argument, which
  turns -0 into +0. erf now returns a zero argument unchanged, and the zero
  branch of erfinv returns the argument instead of its renormalized copy.

No other result changes. math_special_values checks all of it at fp32mp2,
fp32mp2_low and fp32mp2_high, with sinpi at nonzero integers, whose zero
already had the sign of the argument.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…kernel at low and mid accuracy

The one-reciprocal erfcx kernel of erfc, erfcx(a) = s * R(s - 9/16) with
s = 4 / (a + 4), becomes a function of its own,
__internal_fpmp2_erfcx_short_kernel, and erfcx and normcdf use it at the low
and default levels where they used the kernel with the two refined divisions:

- erfcx for 1/2 <= |x| <= 10.0546875, where the fit of the kernel ends;
  from there to 40 it keeps the other kernel.
- normcdf for |x| / sqrt(2) >= 1/2. Its result is now zero below x = -14.219
  instead of -14.5: there the result is under 2^-151, which rounds to zero,
  and up to it the argument of the kernel stays inside the fit.

fpmp2_accuracy::high keeps the kernel with the two divisions, as
__internal_fpmp2_high_erfcx and __internal_fpmp2_high_normcdf, which the
public functions select for fp32mp2_high; its results are the same bits as
before on every range tested. fp64mp2 has one erfcx and one normcdf for every
level. erfc does not change at any level.

erfcx is also restructured, at every level: the exact-argument exp and the
kernel are each written once, shared by the cases that need them (negative
arguments and |x| < 1/2 for the exp; both signs for the kernel). On the
device a warp whose threads take different cases executes every case, so each
extra inlined copy was paid for by all of them.

Against NVIDIA#12641 and the parent commit, which agree for these two functions:

- Host, ns per call on an 11th Gen Intel Core i5-11500T, GCC 15.2
  -O3 -march=x86-64-v3: erfcx on [-3, 10] 102.9 -> 67.0 (1.54x), normcdf on
  [-5, 5] 151.2 -> 109.7 (1.38x). At fpmp2_accuracy::high erfcx is
  103.0 -> 106.8 (0.96x, the price of the shared exp on the host) and normcdf
  is unchanged.
- Device, executed instructions per call on sm_89 for arguments that mix the
  cases: erfcx 35.2 -> 19.4 (1.82x fewer) and 35.2 -> 21.4 at
  fpmp2_accuracy::high (1.65x fewer, same results); normcdf 22.5 -> 20.4.
- Largest error against MPFR over 1M arguments per range, in u^2 = 2^-48:
  erfcx 3.9 -> 7.3 on [-3, 10] and [8, 2^60], normcdf 4.5 -> 7.0 on [-13, 5];
  on the device 6.1 and 7.6. fpmp2_accuracy::high: 3.9 and 4.5, as before.

New test fpmp/math_erfcx.pass.cpp, for the three levels: erfcx across its
ranges and at its ends, normcdf in both tails, in the subnormal range and
where it is exactly 0 or 1.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Review info

Run configuration

  • Source mode: Auto
  • Repository: NVIDIA/cccl
  • Branch: main
  • File: .coderabbit.yaml
  • Commit: 8d33eb1
  • Layers: Repository YAML
  • Review profile: CHILL
  • Plan: Enterprise
  • Run ID: 1c9319f4-5838-45b1-a08f-1be4da937660
Commits

Reviewing files that changed from the base of the PR and between 0013716 and 0d3de05.

Files selected for processing: 8

  • docs/libcudacxx/extended_api/fp/fpmp.rst
  • libcudacxx/include/cuda/__fp/fpmp_math.h
  • libcudacxx/include/cuda/__fp/fpmp_math_impl.h
  • libcudacxx/include/cuda/__fp/fpmp_math_impl_lib.h
  • libcudacxx/include/cuda/__fp/fpmp_math_impl_special.h
  • libcudacxx/include/cuda/__fp/fpmp_math_impl_trig.h
  • libcudacxx/test/libcudacxx/cuda/fp/units/fpmp/math_erfcx.pass.cpp
  • libcudacxx/test/libcudacxx/cuda/fp/units/fpmp/math_special_values.pass.cpp
Files skipped from review as they are similar to previous changes: 2

  • docs/libcudacxx/extended_api/fp/fpmp.rst
  • libcudacxx/include/cuda/__fp/fpmp_math_impl.h

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

rmlarsen and others added 2 commits November 14, 2026 20:41
…elects: fixes from the review of NVIDIA#12568

- llrint / llround: the rounded pair is converted by a helper that also
  takes the pairs at +-2^63 that are in range through their low limb
  (-2^63 + lo, 2^63 - |lo|) and a pair below 2^63 whose rounding carries
  into the high limb. (-2^63, 1) returned LLONG_MIN, (2^63, -2^38) the
  platform's out-of-range sentinel, and (2^63 - 2^39, 2^38) was off by one
  on the device.
- erfcx: the overflow cut-off on the negative side is the last float whose
  result can be finite, with the low limb deciding after the doubling
  (15 floats returned +inf early); for x > 40 the division is by x 2^-64
  and the quotient is scaled back, so that the device's approximate
  reciprocal no longer flushes the result to 0 for x >= 2^127.
- __internal_fpmp2_exp_scales: the exponent fields are formed in unsigned
  arithmetic (the signed difference overflowed for n >= 2); same bits.
- sin, tan, sinpi and erf return a zero argument with its sign only when
  both limbs are zero: a low-accuracy difference can carry a zero high
  limb over a nonzero low one, and its result lost the high limb.
- Tests: TEST_HAS_CUDA_FAST_MATH is tested by value (it is always defined),
  which turns the subnormal checks back on in the ordinary build; the
  subnormal rows of math_cbrt and math_pow's gradual underflow are left to
  the IEEE build; floor(-0.25) must be -1 and the other rounding functions
  -0; new checks for the cases above.
- Docs: boys_f0 is specified for a >= 0.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
fpmp_math_impl_special.h conflicted in erfcx, which this branch restructures
as __internal_fpmp2_erfcx_core<_High>: the three fixes are carried over by
hand (the overflow cut-off at 0x1.2c3cbcp+3 with the check after the
doubling, and the division by x 2^-64 in the x > 40 tail). The sin kernel of
the low and default levels gets the same both-limb test for a zero argument
as the kernels fixed in NVIDIA#12568.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@rmlarsen

Copy link
Copy Markdown
Contributor Author

/ok to test 02b1d83

rmlarsen and others added 7 commits November 14, 2026 21:17
A low-accuracy difference can carry a zero high limb over a nonzero low one
((2^24, 0.25) - 2^24 is (0, 0.25)). The shortcuts that answer a zero
argument for the sake of its sign took such a pair for zero: the fp32mp2
expm1, log1p, sinh and tanh returned it unchanged (0.25 instead of 0.284
for expm1), the fp64mp2 atan and asin likewise, and the fp64mp2 atan2
returned +-0 or +-pi for such a y. They now test both limbs, which gives
these pairs the results they had before the shortcuts were added and keeps
the sign of a true zero.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…-kernels

fpmp_math_impl_exp.h and fpmp_math_impl_hyperbolic.h keep this branch's
side: its fp32mp2 expm1, log1p, sinh and tanh have no zero shortcut to fix
(they take a tiny-argument path by the high limb, which is left as it is,
so the four fp32mp2 checks of the merged test are dropped here). The
fp64mp2 sinh, tanh, asinh, atanh, expm1 and log1p shortcuts that this branch
adds get the same both-limb test as atan, asin and atan2, with checks.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…r zero high limbs by the low limbs; fast-math test guards

- sinh: past e/2 = 2^31 (|x| > 22.2) the term 0.25 / (e/2) is under 2^-63
  of the result. It is dropped and the divisor of the refined division is
  clamped there, as __internal_fpmp2_half_inv_exp does for cosh: dividing
  by a larger e forms subnormal products, and next to the overflow
  threshold a subnormal reciprocal. Same results on every range tested;
  109.9 -> 31.2 ns on [72, 88] and 363.2 -> 31.2 ns on [88.8, 89.4]
  (11th Gen Intel(R) Core(TM) i5-11500T).
- fmax, fmin: the +0 / -0 tie-break applies only when the low limbs are
  equal, so that two pairs with zero high limbs (low-accuracy differences)
  are ordered by their low limbs: fmax((+0, -d), (-0, 0)) returned -d.
- math_invtrig.pass.cpp: the rows with subnormal operands are left to the
  IEEE build, and the smallest normal stands in for the smallest subnormal
  in the special-value tables, when the device compiles with
  --use_fast_math (the CCCL_ENABLE_FASTMATH job of the PR matrix).

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…rmal test rows to the IEEE build

fpmp_math_impl_hyperbolic.h keeps this branch's side: its sinh has no
unclamped reciprocal. fmax and fmin take NVIDIA#12568's order of zero high limbs.

Tests: the PR matrix builds the libcudacxx tests once with --use_fast_math,
which flushes subnormals to zero on the device. In math_erfc, math_erfcx,
math_explog, math_hyperbolic and math_sqrt every table row with a subnormal
operand, or with a result below 2^-78 (whose low limb can be subnormal), is
now left to the IEEE build, as are the checks of subnormal erfc and normcdf
results; the special-value tables of math_explog and math_hyperbolic use
the smallest normal in place of the smallest subnormal there.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…e a loose argument; cast-based double conversion: zero low limb for a non-finite high limb

- expm1, log1p, sinh, tanh, asinh, atanh: the fp32mp2 kernels take the size
  of the argument, and with it the result f(x) = x of a tiny argument and
  the table entry, from the high limb alone. For the pair (0, 0.25), which
  (2^24, 0.25) - 2^24 leaves at fpmp2_accuracy::low, all six returned 0.25
  (expm1 and log1p 12 % off). At the low level an argument with
  |lo| > 2^-20 |hi| is now renormalized before the kernel; a normalized
  pair, a zero of either sign and a limb that is not finite pass through
  unchanged, and the other levels are not touched. Executed instructions
  per call on normalized arguments, host, GCC 15.2 -O2 -mfma: -1.6 % to
  +2.7 %, results bit-identical.
- __fpmp2_from_double, cast-based path (CCCL_FPMP_OPTIMIZED_DOUBLE_TO_FPMP=0):
  a double that is infinite, NaN or beyond FLT_MAX gave a low limb of NaN
  or of the opposite infinity, and with it the fp32mp2_high norm3d, norm4d,
  rnorm3d and rnorm4d, which convert a double result: (inf, -inf) for an
  overflowing norm and (inf, NaN) for the reciprocal norm of zeros. The low
  limb is now zero there, as on the integer path.
- Tests: math_special_values checks the six functions on loose fp32mp2_low
  pairs again, to 1e-9; the new cast_plain_double runs the conversion and
  the four norms with CCCL_FPMP_OPTIMIZED_DOUBLE_TO_FPMP=0, where
  math_sqrt.pass.cpp aborted before.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…rithmetic

Low-accuracy arithmetic does not renormalize, so a difference can carry a
low limb that is not small against its high limb. Three entry paths took
the high limb alone for the value of such a pair:

- atanh: the closed form for |x| = (1, -d) holds to d^2/8 and is meant for
  the d <= 2^-25 of a normalized pair. It ran for any negative low limb
  under a high limb of 1: atanh((1, -0.25)), the pair that
  (2^24, -0.25) - (2^24 - 1) leaves, was 0.9772 instead of 0.9730. A
  larger d now takes the log1p form, as it did before the closed form was
  added.
- erfinv: the zero branch tests the renormalized argument and returned the
  argument's own high limb for the sign of zero, which is 2 for the pair
  (2, -2), a zero. It returns that limb only when both limbs are zero.
- atan2: an operand with a zero high limb over a nonzero low limb was read
  as a zero with the sign of the high limb: atan2(0, (0, -2^-30)) was 0
  instead of pi, atan2((0, -2^-30), -1) was +pi instead of -pi, and two
  such operands gave 0 for every direction. The fp32mp2 function takes
  such an operand for its low limb, behind one test of the bit patterns
  of the high limbs, and the zero-y shortcut of the fp64mp2 function reads
  the sign of the low limb of such an x.

Cost on ordinary arguments, in executed instructions: atan2 295 -> 299 per
warp of 32 calls on sm_89 (ncu) and 314 -> 319 per call on x86-64
(callgrind); atanh and erfinv are unchanged to within one instruction.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…y pairs of atan2, atanh and erfinv

Conflicts:
- fpmp_math_impl_trig.h: the operand promotion of atan2 goes into
  __internal_fpmp2_atan2_core, the entry code that every accuracy level
  shares, so the table-driven kernel and the polynomial kernel both get
  an operand with a zero high limb as its low limb.
- fpmp_math_impl_hyperbolic.h: the d <= 2^-25 guard goes around the closed
  form of __internal_fpmp2_high_atanh, which keeps this branch's
  __fpmp2_high_log1p call. The low and default kernel is not touched: at
  the low level its argument is renormalized before it.

erfinv and the fp64mp2 atan2 shortcut merge as they are.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: In Review

Development

Successfully merging this pull request may close these issues.

2 participants