Skip to content

[Performance] Use thrust::cuda::par_nosync instead of thrust::cuda::par - #79961

Open
sneaxiy wants to merge 2 commits into
PaddlePaddle:developfrom
sneaxiy:thrust_par_nosync
Open

sneaxiy wants to merge 2 commits into
PaddlePaddle:developfrom
sneaxiy:thrust_par_nosync

Conversation

@sneaxiy

@sneaxiy sneaxiy commented Nov 18, 2026 •

Copy link
Copy Markdown
Member

PR Category

Performance Optimization

PR Types

Performance

Description

Replace thrust::cuda::par with thrust::cuda::par_nosync under paddle/ (47 call sites in 23 files), through a new macro PADDLE_THRUST_CUDA_PAR defined in paddle/phi/backends/gpu/gpu_thrust_policy.h. thrust::cuda::par synchronizes the stream after every algorithm call; par_nosync skips that optional sync. HIP paths (thrust::hip::par) are unchanged.

Why removing the sync is safe

  • Algorithms that return a value to the host (reduce, equal, unique/unique_by_key, reduce_by_key, remove/copy_if, min_element, ...) still perform their mandatory synchronization under par_nosync.
  • For the rest (transform, fill, sort, scan, lower_bound, sequence, ...), the results are either consumed only by later work on the same stream, or read on the host after a stream-ordered D2H copy followed by Wait() / cudaStreamSynchronize.
  • Temporary storage is stream-ordered (Paddle's ThrustAllocator and DenseTensor), or released with cudaFree, which synchronizes.

Two places that relied on the implicit sync are fixed

Paddle's default stream is blocking, but user-created streams are cudaStreamNonBlocking, so work on the legacy default stream is not ordered with them:

  • gpu/unique_kernel.cu: inv_loc[0] = 0 was written from the host through thrust::device_ptr, which runs on the legacy default stream and could race with the preceding adjacent_difference. It is now written with thrust::fill on the kernel's stream.
  • sparse/gpu/sparse_utils_kernel.cu, sparse/gpu/mask_kernel.cu (both the 2D and 3D paths): GetBatchSizes and ConvertCsrCrowsToCooRows were launched without a stream around a thrust::exclusive_scan. They now launch on dev_ctx.stream().

CUDA 11.x compatibility

  • par_nosync requires Thrust >= 1.16. PADDLE_THRUST_CUDA_PAR is thrust::cuda::par_nosync when THRUST_VERSION >= 101600 and falls back to thrust::cuda::par otherwise, so builds with the CUDA 11.x toolkit Thrust (1.15) keep compiling with the old behavior.

Testing

  • Full build with -DWITH_GPU=ON -DWITH_SHARED_PHI=ON -DCUDA_ARCH_BIN=100 on CUDA 13.2 (aarch64) succeeds; paddle.utils.run_check() passes on a single GPU.
  • New test/legacy_test/test_thrust_nosync_stream.py: runs paddle.unique (with return_inverse, flat and axis=0), CSR paddle.sparse.mask_as (2D/3D) and 3D CSR->COO on a user-created non-blocking stream and compares with numpy.
  • Passed: test_thrust_nosync_stream, test_unique, test_sparse_mask_as_op, test_sparse_utils_op, test_sparse_softmax_op, test_sparse_conv_op, test_sparse_pooling_op, test_sparse_slice_op, test_median, test_nanmedian, test_cumsum_op, test_argsort_op.

是否引起精度变化

否

🤖 Generated with Claude Code

thrust::cuda::par synchronizes the stream after every algorithm call.
None of these call sites need it: results are consumed by later work on
the same stream, or read on the host after an explicit stream-ordered
copy plus Wait/cudaStreamSynchronize, or come from algorithms whose
mandatory sync is kept by par_nosync (reduce, equal, unique_by_key,
reduce_by_key, remove, min_element, ...).

Two places relied on the implicit sync for correctness on non-blocking
streams, and are fixed here:
- unique_kernel.cu: inv_loc[0] was written from the host through a
  thrust::device_ptr, which runs on the legacy default stream. Use
  thrust::fill on the kernel's stream instead.
- sparse mask/sparse_utils kernels: GetBatchSizes and
  ConvertCsrCrowsToCooRows were launched without a stream. Launch them
  on dev_ctx.stream().

Co-Authored-By: Claude Code <noreply@anthropic.com>
@Paddle-Review-Bot

Paddle-Review-Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Paddle-Bot Review Board (review完成)

序号 位置 优先级 规则来源 状态
1 par_nosync 依赖 Thrust 1.16,CUDA 11.x 构建失败 P1 仓库规则:基础评审规则(构建配置与设备能力保护) ✅
2 稀疏 mask 2D 路径仍在默认流启动 P2 仓库规则:基础评审规则(kernel 设备与 stream)
默认规则
✅

可维护性建议(总分 89.7 / 100,只给建议,不影响评审结论)

序号 位置 维度 状态
1 stream 顺序修复缺少非默认流回归测试 测试策略 ✅
各维度得分(仅评本 PR 改动,点击展开)
86 API 表面一致性
86 中层优雅度
86 未完成迁移
86 测试策略
1
86 命名质量
86 底层优雅度
87 设计一致性
88 跨模块架构
88 高层优雅度
90 约定偏离
92 错误处理一致性
92 契约一致性
92 抽象适配度
92 逻辑清晰度
92 类型安全
93 依赖健康度
93 包组织结构
95 初始化耦合
95 AI 生成债务
— 鉴权一致性

Powered by Nyanpasu claude with Opus 5.5 默认推理级别, please check the suggestions carefully.

Comment thread paddle/phi/common/transform.h Outdated
op);
#else
thrust::transform(thrust::cuda::par.on(dev_ctx.stream()),
thrust::transform(thrust::cuda::par_nosync.on(dev_ctx.stream()),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 thrust::cuda::par_nosync 从 Thrust 1.16 才提供,但当前构建仍允许 CUDA 11.7/11.8,这些配置下会编译失败。

  • cmake/configure.cmake:116 的最低要求仍是 CUDA >= 11.7;setup.py / python/setup.py.in 仍有 "11.8" 的依赖表,tools/dockerfile/ubuntu20_dev.sh 默认 ref_CUDA_MAJOR=11.8。
  • cmake/third_party.cmake:484-493 只在 CUDA >= 12.0 且 WITH_SHARED_PHI 时引入 CCCL;CUDA 11.7/11.8 只替换了 CUB(cmake/external/cub.cmake:27-29),Thrust 仍是 toolkit 自带的 1.15.x,其中没有 par_nosync。
  • PR 描述里也提到 CUDA 11.x 不支持,但代码里没有版本保护,本 PR 改到的 47 处都会受影响。

建议二选一:

  1. 在公共头里统一按版本选择,各调用点改用这个宏,例如:
    #if THRUST_VERSION >= 101600
    #define PADDLE_THRUST_CUDA_PAR thrust::cuda::par_nosync
    #else
    #define PADDLE_THRUST_CUDA_PAR thrust::cuda::par
    #endif
    注意:回退到 par 后,unique_kernel.cu 和稀疏 kernel 里补上的 stream 顺序修复仍然有效。
  2. 明确放弃 CUDA 11.x,同步把 configure.cmake 的最低版本提到 12.0,并清理 11.8 相关的打包配置。

Comment on lines +285 to +288
ConvertCsrCrowsToCooRows<IntT><<<config.block_per_grid,
config.thread_per_block.x,
0,
dev_ctx.stream()>>>(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 这里给 3D 路径补上了 dev_ctx.stream(),但同文件 2D 路径 MaskCsr2DGPUKernel 的同一个 kernel(mask_kernel.cu:191-193)仍在 legacy default stream 上启动:

ConvertCsrCrowsToCooRows<IntT>
    <<<config.block_per_grid, config.thread_per_block.x>>>(...);

它后面的 GpuMemcpyAsync 和 MaskKernel(第 206 行)都在 dev_ctx.stream() 上,并且会读取这个 kernel 写出的 coo_rows_data。当 dev_ctx.stream() 是用户创建的 cudaStreamNonBlocking stream 时,这两者之间没有顺序保证,就是 PR 描述里要修的那类竞争。建议按本处的写法补成 <<<config.block_per_grid, config.thread_per_block.x, 0, dev_ctx.stream()>>>。

inv_loc_data_dev[0] = 0; // without device_ptr, segmentation fault
// Set inv_loc[0] = 0 on dev_ctx.stream() so it is ordered after the
// adjacent_difference above, which no longer synchronizes the stream.
thrust::fill(exec_policy, inv_loc_data_ptr, inv_loc_data_ptr + 1, 0);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 可维护性建议(测试策略,不阻塞合入):这个 PR 修复了两处依赖隐式同步的 stream 顺序问题(这里的 inv_loc[0],以及稀疏 mask / csr->coo 的 kernel 启动),但没有配套测试。现有 test/legacy_test/test_unique.py 和 test_sparse_mask_as_op.py 都只在默认 stream 上运行,旧代码在默认 stream 下同样能通过,以后有人回退这些修改也测不出来。建议参照 test/legacy_test/test_cuda_stream_event.py,在 paddle.device.cuda.stream_guard(paddle.device.cuda.Stream()) 下补一个用例:用较大输入调用 paddle.unique(..., return_inverse=True) 和 3D CSR 的 mask_as / to_sparse_coo,并与 numpy 结果对比。

- Add PADDLE_THRUST_CUDA_PAR in paddle/phi/backends/gpu/gpu_thrust_policy.h.
  It is thrust::cuda::par_nosync for Thrust >= 1.16 and falls back to
  thrust::cuda::par otherwise, so CUDA 11.7/11.8 builds (toolkit Thrust
  1.15) keep compiling. Use it at all call sites.
- Launch ConvertCsrCrowsToCooRows on dev_ctx.stream() in
  MaskCsr2DGPUKernel too.
- Add test_thrust_nosync_stream.py, which runs unique (return_inverse),
  CSR mask_as (2D/3D) and 3D CSR->COO on a user-created non-blocking
  stream and compares against numpy.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@sneaxiy
sneaxiy requested a review from ForFishes November 19, 2026 04:22

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants