Skip to content

regex-syntax: Optimize SimpleCaseFolder - #1396

Open
Keats wants to merge 2 commits into
rust-lang:masterfrom
Keats:master
Open

Keats wants to merge 2 commits into
rust-lang:masterfrom
Keats:master

Conversation

@Keats

@Keats Keats commented Sep 17, 2026

Copy link
Copy Markdown

In giallo, the PHP grammar (https://github.com/shikijs/textmate-grammars-themes/blob/4cb1625ad87a07fb4674fe97b921a8f756f4694b/packages/tm-grammars/grammars/php.json) has a lot of patterns like (?i)[_a-z\x7F-\x{10FFFF}].

With master case folding, it would iterate every codepoint and be generally slow. Replacing it with an iterator that looks at the table directly is much faster. The code is also much simpler, I'm not sure if I'm missing something tbh.

Benchmark results with the added bench from that PR:

master:

test parse_full_folding   ... bench:   2,767,636.50 ns/iter (+/- 321,494.26)
test parse_huge           ... bench:       1,175.10 ns/iter (+/- 9.38)
test parse_medium1        ... bench:       4,395.45 ns/iter (+/- 43.96)
test parse_medium2        ... bench:       3,892.35 ns/iter (+/- 25.34)
test parse_medium3        ... bench:      12,665.54 ns/iter (+/- 72.62)
test parse_simple1        ... bench:       1,126.57 ns/iter (+/- 26.19)
test parse_simple2        ... bench:       2,204.90 ns/iter (+/- 29.84)
test parse_simple_folding ... bench:         762.16 ns/iter (+/- 10.08)
test parse_small1         ... bench:      12,715.30 ns/iter (+/- 116.87)

this branch:

test parse_full_folding   ... bench:      36,826.69 ns/iter (+/- 688.75)
test parse_huge           ... bench:       1,159.02 ns/iter (+/- 9.32)
test parse_medium1        ... bench:       4,360.44 ns/iter (+/- 38.80)
test parse_medium2        ... bench:       3,897.28 ns/iter (+/- 34.22)
test parse_medium3        ... bench:      12,760.71 ns/iter (+/- 370.60)
test parse_simple1        ... bench:       1,125.55 ns/iter (+/- 14.01)
test parse_simple2        ... bench:       2,095.07 ns/iter (+/- 24.99)
test parse_simple_folding ... bench:         785.47 ns/iter (+/- 7.44)
test parse_small1         ... bench:      12,747.68 ns/iter (+/- 223.23)

So about 70x faster for that pattern and no changes for simpler ones.

Kinda related: #893

@Keats Keats changed the title Optimize SimpleCaseFolder regex-syntax: Optimize SimpleCaseFolder Sep 21, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant