refactor riscv gemm microkernel by nihui · Pull Request #6752 · Tencent/ncnn

nihui · 2026-05-28T16:39:27Z

No description provided.

tencent-adm · 2026-05-28T16:39:46Z

Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

codecov-commenter · 2026-05-28T16:45:01Z

Codecov Report

❌ Patch coverage is 94.05265% with 183 lines in your changes missing coverage. Please review.
✅ Project coverage is 95.15%. Comparing base (9f9d4ec) to head (5e32cac).
⚠️ Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
src/layer/riscv/multiheadattention_riscv.cpp	81.04%	58 Missing ⚠️
src/layer/riscv/gemm_fp16sa.h	95.25%	57 Missing ⚠️
src/layer/riscv/sdpa_riscv.cpp	88.53%	29 Missing ⚠️
src/layer/riscv/gemm_fp16s.h	93.68%	25 Missing ⚠️
src/layer/riscv/gemm_bf16s_fp16s.h	87.93%	7 Missing ⚠️
src/layer/riscv/gemm_riscv_zfh.cpp	98.79%	7 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #6752      +/-   ##
==========================================
- Coverage   95.19%   95.15%   -0.05%     
==========================================
  Files        1457     1456       -1     
  Lines      493714   495505    +1791     
==========================================
+ Hits       469996   471482    +1486     
- Misses      23718    24023     +305

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

nihui · 2026-05-30T08:23:09Z

@codex review

Copilot

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

nihui · 2026-05-30T08:25:45Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e0a6655f26

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copilot

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.

github-actions · 2026-10-04T10:04:46Z

Please enable github action in YOUR FORKED REPO to make code-format workflow work

…ME2 branch 合并上游对 riscv GEMM 微内核与 SDPA 的重构（Tencent#6752，含新的 sdpa_riscv 子层实现）。冲突文件以上游新版为基，本分支的增量（IME2 Gemm 接管块、M=1 解码快路径）在随后的提交中移植。 Merge the upstream riscv GEMM microkernel / SDPA refactor (Tencent#6752, which adds its own sdpa_riscv sub-layer implementation). The conflicting files take the upstream version as the base; this branch's additions (the IME2 Gemm takeover block and the M=1 decode fast path) are ported in the following commits.

…he upstream refactor 在上游 Tencent#6752 重构后的基线上保留本分支的 SDPA 增量： * forward_rvv_decode —— M=1 解码快速路径（QK^T 逐位置点积 + 流式 P·V + RVV 向量化 exp），解决解码随上下文长度塌陷的问题（507 token 上下文：5.22 -> 8.07 tok/s 于该改动当时）； * forward_ime2_prefill / ime2_available —— SpacemiT K3 IME2 的 flash 式预填充路径； * 与之配套的解码分段计时（NCNN_SDPA_DECODE_PROF）与 KV cache 管理。注：上游 Tencent#6752 同时新增了一版 sdpa_riscv（子层 qk_gemm/qkv_gemm/qk_softmax 实现）。本分支保留了自研实现，因为它额外提供 M=1 解码快路径与 IME2 预填充，并已通过三腿逐字节闸门；若上游更希望采用子层方案，可将本文件的解码路径作为增量移植到该方案之上（接口已兼容）。 Keep this branch's SDPA additions on top of the upstream refactor (Tencent#6752): the M=1 decode fast path (per-position dot product + streaming P·V + vectorised exp, which stops decode collapsing with context length), the IME2 flash prefill path, and the accompanying decode segment timers and KV-cache handling. Upstream Tencent#6752 also added its own sdpa_riscv (sub-layer based). This branch keeps the standalone implementation because it additionally provides the M=1 decode path and IME2 prefill, and it passes the three-leg byte-identical gate. If upstream prefers the sub-layer design, the decode path here can be ported onto it (the interfaces are compatible, as verified by building against the refactored base).

合并远端自动格式化提交；冲突文件取本分支版本，因为它已包含上游 Tencent#6752 的重构，若取远端版本会把该重构回退掉。格式由上游 CI 统一处理。 Merge the remote code-format commit. The conflicted file keeps this branch's version because it already contains the upstream Tencent#6752 refactor; taking the remote side would revert it. Formatting is left to the upstream CI.

wip

2b8db45

github-actions Bot added the riscv label May 28, 2026

nihui added 2 commits May 30, 2026 13:37

w

24c1b0f

w

9991ede

nihui marked this pull request as ready for review May 30, 2026 08:21

nihui changed the title ~~[WIP] refactor riscv gemm microkernel~~ refactor riscv gemm microkernel May 30, 2026

nihui requested a review from Copilot May 30, 2026 08:23

Copilot started reviewing on behalf of nihui May 30, 2026 08:23

View session

apply code-format changes

e0a6655

nihui closed this May 30, 2026

nihui reopened this May 30, 2026

Copilot AI reviewed May 30, 2026

nihui requested a review from Copilot May 30, 2026 08:25

Copilot started reviewing on behalf of nihui May 30, 2026 08:26

View session

chatgpt-codex-connector Bot reviewed May 30, 2026

View reviewed changes

Comment thread src/layer/riscv/gemm_riscv.cpp Outdated

Copilot AI reviewed May 30, 2026

View reviewed changes

Comment thread src/layer/riscv/gemm_riscv_zfh.cpp Outdated

nihui and others added 2 commits June 6, 2026 13:13

w

4b94957

apply code-format changes

fce8aa4

nihui closed this Jun 6, 2026

nihui reopened this Jun 6, 2026

nihui and others added 6 commits June 6, 2026 14:48

w

9f1b262

c

f2826ea

Merge branch 'master' into opt-riscv-gemm

bcb4355

up

3d73be9

riscv mha and sdpa, opt gemm wq int8

934b67f

apply code-format changes

5e32cac

nihui merged commit 5e33816 into Tencent:master Oct 4, 2026
50 of 52 checks passed

HougeLangley mentioned this pull request Oct 8, 2026

riscv/arm Gemm: a bare layer with an fp32 input and use_fp16_storage (default) reads fp16-packed BT_data as fp32 -- SIGSEGV or silently wrong results #7050

Closed

Navigation Menu

Repository navigation

refactor riscv gemm microkernel - #6752

refactor riscv gemm microkernel#6752
nihui merged 12 commits into
Tencent:masterfrom
nihui:opt-riscv-gemm

nihui commented May 28, 2026

Uh oh!

tencent-adm commented May 28, 2026

Uh oh!

codecov-commenter commented May 28, 2026 •

edited

Loading

Uh oh!

nihui commented May 30, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

nihui commented May 30, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

github-actions Bot commented Oct 4, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

nihui commented May 28, 2026

Uh oh!

tencent-adm commented May 28, 2026

Uh oh!

codecov-commenter commented May 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

nihui commented May 30, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

nihui commented May 30, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

github-actions Bot commented Oct 4, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

codecov-commenter commented May 28, 2026 •

edited

Loading