Repository navigation
Conversation
Contributor
|
Hi @kevodwyer, thanks a lot for your interest in this project and for sharing this experiment! I'll have a closer look at it sometime soon, a few top of head:
|
kevodwyer
added a commit
to kevodwyer/endive
that referenced
this pull request
Oct 27, 2026
Review on bytecodealliance#256 asked how this compares to endive's Redline backend (Cranelift, native), not just to main. It runs automatically when the quickjs4j checkout has the redline module from roastedroot/quickjs4j#209, and is skipped otherwise. All three configurations build from one quickjs4j checkout, since that PR also rebuilds the plugin wasm. On JDK 27 with Panama, per iteration: Redline 17.5 ms, split 36.6 ms, unsplit 274.0 ms. Redline is 2.1x faster than splitting, which closes ~93% of the gap from unsplit bytecode to native. The gain from this branch is therefore to the portable pure-JVM path, which needs no per-target native code and no FFI runner. Redline has to be measured on its intended runner: below Java 25 the Panama one cannot be built, and jffi's per-call overhead made it look 4x slower than splitting instead. So its endive version is chosen from the running JDK, the script builds endive's redline modules itself, and the report's caveats follow the runner actually selected. Also fixes a find that could locate core's test-jar instead of the jar under test, and a "% of distance to native" that is meaningless when native is slower.
andreaTP
reviewed
Oct 27, 2026
|
|
||
| Two things to know when reading its numbers: | ||
|
|
||
| - Redline is built against a **released** endive (`REDLINE_ENDIVE_VERSION`, |
Contributor
There was a problem hiding this comment.
Redline should be used from main, we fixed a few bugs that would prevent good performance since the last release.
We are going to release sometime soon, but is not out yet.
Author
There was a problem hiding this comment.
Ran the benchmark again with Redline built from main - results in line with previous run. Where results did change was in adding a sustained-execution workload:
split 7.2x
Redline 2.8x faster than splitting
Contributor
There was a problem hiding this comment.
thanks for the double check and the numbers!
Author
|
Have run benchmark using JDK 27 with Panama, per iteration: |
Contributor
This is approximately my understanding too, please note that results may vary a lot depending on the workload, here we are benchmarking against one single module. |
A Wasm function that compiles to more than HUGE_METHOD_LIMIT bytes of JVM bytecode is past the size HotSpot will compile, so it is never JITed and runs in the JVM's own bytecode interpreter no matter how hot it gets. Nothing warns about it. Real modules hit this hard: QuickJS's JS_CallInternal is 56671 bytes and is among the hottest code in the program. That ceiling is the one dispatch methods are already sized against a few lines above, so this reuses the same constant and the same knob rather than inventing a second notion of "too big to JIT". Split such functions into JIT-eligible pieces, in three escalating steps: * Linear chunking cuts at labels where the operand stack is provably empty (ASM's own data-flow analysis, not a guess) and no branch spans the cut. * Switch splitting additionally lets a br_table's targets become chunk boundaries, resolving far targets through per-chunk stubs. A single switch's targets all share one source position, which otherwise makes every candidate look span-blocked. * Trampoline splitting handles what neither can: bodies dominated by a loop. A loop's back-edge is a goto, and a goto cannot cross a method boundary, so no position inside a loop is ever a legal linear cut -- and measured on real code that is 97.5% of JS_CallInternal and 98.2% of printf_core. Instead of cutting around loops, inter-chunk control flow stops using gotos: every local lives in a long[] frame, each chunk returns the index of the next chunk, and a driver loop in the entry method dispatches. A back-edge is then just another PC value. Because a chunk stores PC=DONE on entry, an ordinary Wasm return stays an ordinary areturn and no opcode emitter changes. A function that cannot be split safely is left exactly as it was: compiled to a single method, still not JITable, and silent. That case is a missed optimisation, not a failure, so it must not be pushed to the interpreter and must not fail a build running with InterpreterFallback.FAIL. The threshold is clamped to stay well under the 65535-byte ceiling on a JVM method's Code attribute, so raising endive.hugeMethodLimit cannot lead the planner to emit a chunk the class writer then rejects. Also routes functions with very many try_table handlers to the interpreter rather than letting ASM's exception-table handling stall the build. compiler-tests: 34110 pass.
Trampoline chunks boxed and unboxed every local in the function on every transition. That is the wrong cost to pay exactly where trampolining is most valuable: a dispatch-loop case body typically touches a handful of locals, and JS_CallInternal has 48, so each pass through the loop moved all 48 twice. The long[] frame is the single source of truth and the JVM slots are only a per-chunk cache of it, so a chunk need only load the locals it reads or writes and store back the ones it writes. A local the chunk never mentions keeps its value in the frame untouched. Written locals are loaded too, not just stored: a write that happens on only one path through the chunk would otherwise leave the slot undefined, and the shared epilogue stores unconditionally. Loading first makes the worst case a store of the value that was already there. Only LOCAL_GET, LOCAL_SET and LOCAL_TEE touch local slots (LOCAL_TEE dups the stack value and delegates to LOCAL_SET), so the read/write sets are exact rather than approximated, and no dataflow fixpoint is needed. QuickJS throughput improves 76ms -> 33ms per iteration, taking the total gain over an unsplit build to 8.2x (270ms -> 33ms), reproducible across runs.
The planner reserved bytes for every local in the function when sizing each chunk's prologue and epilogue. That was written when chunks synced all of them; they now sync only the locals they touch, so the reserve sat far above the real cost and shrank the budget enough to split functions more finely than needed. Reserve optimistically while seeding, then cost each resulting chunk exactly from its own touched/written sets. A chunk that still comes out too big is subdivided at the last candidate that fits -- packing the leading piece full rather than halving it -- instead of costing the whole function its split. Because subdividing can strand a branch that used to be internal to the chunk, the branch-lands-on-a-chunk-start repair is re-run after every subdivision, so both invariants hold together. Sizing alone is not enough, though. Whether planning succeeds also depends on where the boundaries happened to land: different cuts make different branches cross chunks, and if one of those lands on a position with a non-empty operand stack -- which can never begin a chunk -- the function was written off as unsplittable. Widening the budget shifted the boundaries and, on Boxedwine, gained one function while losing two for exactly that reason. So a failure at one budget is not proof the function cannot be split: retry at progressively tighter budgets, which lands the cuts differently. Boxedwine splits 20 functions with 11 left interpreted, against 18/13 before. QuickJS is unchanged at 17 split, 0 interpreted.
…aries A chunk boundary is a method boundary, so a label with anything pending on the operand stack could not begin a chunk -- and a branch landing on such a label disqualified the whole function. That was the single most common reason splitting still failed, and it made planning fragile: moving the boundaries changed which branches crossed chunks, so a function could stop splitting purely because the seed budget shifted. Such labels are now usable. The verifier already guarantees the stack at a branch matches its target, so a transition spills those values into slots after the PC in the long[] frame and the destination reloads them before its first instruction. Spilling also leaves the stack empty at every exit, which is what keeps one shared epilogue valid for all of them. ASM's analysis already gives the exact depth and types at each label, so no new analysis is needed. References still cannot begin a chunk -- they have no representation in a long[] -- and depth is capped. Boxedwine now splits 21 functions with 10 interpreted (was 20/11); func 4413, 28028 bytes, splits for the first time. QuickJS is unchanged at 17/0.
A function with any try_table was declined outright. It need not be: a JVM exception-table entry names a protected range and its handler, and the only real requirement is that they -- and the visitTryCatchBlock declaring them -- end up in the same method. So each try_table contributes a region no chunk boundary may fall inside, exactly like the existing branch-span rule. Linear and switch chunking still decline try_table; only the trampoline takes it, since that is where the remaining oversized functions are. Functions whose protected ranges blanket the body have nowhere left to cut and still fall back, which is the right answer: measured on Boxedwine, eight of its try_table functions have 85-98% of the body inside protected ranges, leaving runs of only 61-270 instructions. Boxedwine now splits 22 functions with 9 interpreted (was 21/10); func 2068 splits for the first time. QuickJS is unchanged at 17/0, 577 chunks.
Demonstrates what splitting oversized functions is worth, on a real workload rather than a synthetic one: QuickJS via quickjs4j, whose interpreter dispatch loop JS_CallInternal compiles to ~56KB in a single method and so was never JIT-compiled at all. run.sh builds quickjs4j twice from identical, unmodified upstream source -- once against endive at this branch's merge-base with main, once against this branch -- and measures both. The only variable is which endive produced the precompiled bytecode in the jar, which is also the point: consumers get this by rebuilding, with no changes of their own. Results are reported per window, not as one average, because the effect is a JIT effect: the baseline is flat because a method past HugeMethodLimit is never compiled however hot it gets, while the split build improves as HotSpot compiles the chunks. A single number would hide the shape that shows the cause. The harness also asserts both builds compute the same checksum, so a timing difference cannot come from doing different work. Emits a Markdown table and an SVG chart, and the README gives two ways to verify the mechanism directly via the compiler's own logs and -XX:+PrintCompilation. Needs only a JDK, Maven and Python 3. Measured 269.8 -> 31.7 ms/iter (8.5x) on an idle M2 / GraalVM CE 22, and reproduced within 0.8% on a second run.
Review on bytecodealliance#256 asked how this compares to endive's Redline backend (Cranelift, native), not just to main. It runs automatically when the quickjs4j checkout has the redline module from roastedroot/quickjs4j#209, and is skipped otherwise. All three configurations build from one quickjs4j checkout, since that PR also rebuilds the plugin wasm. On JDK 27 with Panama, per iteration: Redline 17.5 ms, split 36.6 ms, unsplit 274.0 ms. Redline is 2.1x faster than splitting, which closes ~93% of the gap from unsplit bytecode to native. The gain from this branch is therefore to the portable pure-JVM path, which needs no per-target native code and no FFI runner. Redline has to be measured on its intended runner: below Java 25 the Panama one cannot be built, and jffi's per-call overhead made it look 4x slower than splitting instead. So its endive version is chosen from the running JDK, the script builds endive's redline modules itself, and the report's caveats follow the runner actually selected. Also fixes a find that could locate core's test-jar instead of the jar under test, and a "% of distance to native" that is meaningless when native is slower.
Review asked for Redline from main, since it has post-release fixes that matter to its performance. The previous run built it from this branch -- equivalent, as the branch does not touch the Redline path, but not what was asked and not the cleaner attribution. Redline's modules are now built from a worktree at the same revision as the `before` row, and it is measured before `after` installs this branch's endive over it. Previously it ran last and silently picked up whatever HEAD had installed. On JDK 27 with Panama, per iteration: Redline 19.1 ms, split 36.6 ms, unsplit 266.5 ms -- Redline 1.9x faster than splitting, which closes ~93% of the gap to native. Unchanged from the branch-built figure, as expected.
Splitting needs asm-analysis, which nothing in endive used before, and it
arrives on the bridge's exec-maven-plugin classpath only transitively from
build-time-compiler. The plugin contributes asm and asm-tree itself, nearer the
root of its graph, so those win: with the bump to 9.11.1 the classpath ended up
running 9.11.1 analysis code against 9.10.1 core, and died resolving
LimitExceededException, which exists only in the newer.
Pin asm, asm-tree, asm-analysis and asm-commons there to ${asm.version}.
Reactor and compiler-tests pass on JDK 27 with ASM 9.11.1: 34111 tests.
Review noted that results may vary a lot by workload, and that this benchmarks a single module. The existing workload builds a fresh engine every iteration, so it includes JavaScript compilation and gives natively compiled code the advantage of no warm-up to amortise. The second workload uses one long-lived engine with the script compiled once, so the timed loop is execution alone: recursion, Collatz and loop arithmetic, which is what the dispatch loop actually spends its time on. Per iteration on JDK 27, compile+exec then sustained: before 271.8 ms 829.4 ms after 36.7 ms 114.4 ms 7.4x / 7.2x redline 17.5 ms 40.4 ms 2.1x / 2.8x faster than after The speedup holds across both shapes, so it is the dispatch loop rather than the parser. Redline's lead widens under pure execution, which makes the compile+exec figure the one flattering to the bytecode path, not the reverse.
kevodwyer
force-pushed
the
perf/jit-eligible-chunks
branch
from
October 28, 2026 10:39
9301fd2 to
1ebe205
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR is the result of an experiment a few months back with the assistance of Claude to improve the performance of endive when running an emulator I contribute to. QuickJS4J shares similar innards and so was a good stepping-stone.
I can see from the comments in this issue that the project was thinking of doing something similar so the changes reflect completing an idea rather than introducing a new one - HUGE_METHOD_LIMIT documented as "methods exceeding this get degraded optimization". This change applies that same, already-accepted reasoning to function bodies.