Skip to content

MagCache skips state advancement on one-block cache hits #15339

Open

Description

@TheNetherWatcher

While testing Diffusers caching on CPU with tiny random models, I found a lifecycle bug in the one-block MagCache path. A cache hit bypasses the tail hook because the head and tail wrap the same module. The step counter stays at 1, later magnitude ratios aren't consumed on schedule, and the configured final reset is missed.

Reproduces on main at 2bb9468, Python 3.12.13 / PyTorch 2.15.0, macOS arm64 CPU. No weights or GPU needed:

import torch
from diffusers import MagCacheConfig, WanTransformer3DModel

torch.manual_seed(0)
torch.set_num_threads(1)
model = WanTransformer3DModel(
    patch_size=(1, 2, 2), num_attention_heads=2, attention_head_dim=12,
    in_channels=4, out_channels=4, text_dim=16, freq_dim=32,
    ffn_dim=32, num_layers=1, rope_max_seq_len=32,
).eval()
model.enable_cache(MagCacheConfig(
    num_inference_steps=4, retention_ratio=0.0,
    mag_ratios=[1.0, 1.0, 2.0, 1.0],
))
inputs = dict(
    hidden_states=torch.randn(1, 4, 2, 4, 4),
    encoder_hidden_states=torch.randn(1, 4, 16), timestep=torch.tensor([900]),
)
computed = []
handle = model.blocks[0].attn1.register_forward_hook(lambda *args: computed.append(True))
with torch.no_grad():
    for step in range(4):
        before = len(computed)
        model(**inputs)
        print(step, len(computed) > before)
handle.remove()

Actual full-compute decisions: True, False, False, False. Expected: True, False, True, False, since ratio 2 at step 2 exceeds the default error threshold and should force a refresh. The same thing happens through a tiny WanPipeline call. The step counter goes [1, 1, 1, 1] on main instead of [1, 2, 3, 0].

I have a local fix: mark the one-block head and let its wrapped tail do the state advancement on cache hits, keeping the residual locally for the final reset. The tail doesn't run the transformer block on this path. Three new regression tests fail before the fix and pass after, including a comparison against an uncached Wan model at the required refresh. The hook suite passes (22 passed, 40 accelerator skips), 960 unaffected outputs match the original bit-for-bit, and make quality / make fix-copies pass.

This is a correctness fix, not a speedup. CPU pipeline timings go 1.18→1.31 ms at 32², 1.23→1.42 ms at 64², 1.33→1.59 ms at 128², because the fixed version runs the refresh that was being wrongly skipped. I haven't checked a released one-block checkpoint or output quality.

I checked #14027 (dual-transformer step counts) and #15038 (Qwen-Image 2.1 shape transitions); neither covers this. Would a focused fix with these tests be welcome? I'll wait for a go-ahead before opening a PR.

Disclosure: an AI coding agent helped with the investigation, fix, tests and this write-up. I've reviewed it and will review every changed line before any PR.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions