Repository navigation
Fix quadratic buffer growth in lossy WebP Vp8BitWriter - #3207
Merged
JimBobSquarePants merged 2 commits intoOct 9, 2026
Merged
JimBobSquarePants merged 2 commits into
JimBobSquarePants merged 2 commits into
Conversation
Vp8BitWriter.BitWriterResize compared the needed size against a readonly maxPos field that was initialized to 0 and never updated, so the early return never fired and ResizeBuffer always grew from a base of 0. The partition buffer was therefore reallocated and copied every time the output crossed a 1 KiB boundary (the first call even shrank the expected-size buffer to 1 KiB), making allocations quadratic in the encoded size: a 4000x3000 lossy Quality=90 encode allocated ~24 GiB. Compare against and grow from the current buffer length instead, which mirrors libwebp updating max_pos after each resize and matches Vp8LBitWriter. Encoded output is unchanged.
4 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Prerequisites
Description
Fixes #3206
Vp8BitWriter.BitWriterResizecompared the needed size againstprivate readonly int maxPos, which is set to0in the constructor and never updated. The early return therefore never fired andResizeBuffer(0, neededSize)computed the new size from a base of 0, i.e.neededSizerounded up to the next KiB. As a result the partition buffer was reallocated and copied every time the encoded output crossed a 1 KiB boundary (the very first flush even shrank the expected-size buffer down to 1 KiB), so allocations were quadratic in the encoded size.libwebp updates
bw->max_pos = new_sizeafter resizing (src/utils/bit_writer_utils.c), andVp8LBitWriter.BitWriterResizealready grows fromthis.Buffer.Length. This PR removes the deadmaxPosfield and compares against / grows fromthis.Buffer.Length, so the buffer only reallocates when it is actually full and grows geometrically (1.5x, rounded up to the next KiB).Flushis the only caller: it requestsrun + 1bytes and then writes exactlyrun + 1bytes starting atpos, so checkingpos + extraSize <= Buffer.Lengthguarantees capacity for every write.NumBytes(pos) is unchanged, soWriteToStream/WriteToBufferemit the same bytes.Tests
Vp8BitWriterTests:BitWriterResize_WithinCapacity_DoesNotReallocate: a resize request that fits the current buffer keeps the same array instance.BitWriterResize_GrowsBufferGeometrically: writes 256 KiB through the writer and counts buffer replacements (12 with the fix, 256 before), and checksBuffer.Length >= NumBytes.mainand pass with the fix.dotnet test -c Release --filter "FullyQualifiedName~Webp": 444 passed, net10.0).Before / after (lossy, Quality = 90, default method, .NET 10, Release, Windows 11 x64;
GC.GetTotalAllocatedBytes(true)around a singleSave):Repro from #3206:
Noisier image (gradient + ±64 per-channel noise, larger output):