You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Flux Control and HunyuanVideo LoRA converters swap the scale/shift halves of lora_A in final_layer #15305
final_layer.adaLN_modulation.1 in the original checkpoints orders its output (shift, scale), and diffusers' norm_out.linear orders it (scale, shift), so the converters swap the two halves. For a LoRA only lora_B should be swapped: its rows are the layer's outputs. lora_A is (rank, in), so swapping its halves reorders the rank dimension and the converted delta is wrong.
Two converters in loaders/lora_conversion_utils.py swap both:
_convert_bfl_flux_control_lora_to_diffusers: swap_scale_shift is applied for every lora_key, including lora_A (weight and bias).
_convert_hunyuan_video_lora_to_diffusers: remap_norm_scale_shift_ swaps every final_layer.adaLN_modulation.1 key, lora_A included.
The kohya Flux converter already swaps only lora_B. This is the same class of bug #15276 fixed for Flux2.
Reproduction
importtorchfromdiffusers.loaders.lora_conversion_utilsimport_convert_bfl_flux_control_lora_to_diffusersRANK, INNER=4, 3072torch.manual_seed(0)
classFillingDict(dict):
# dummy tensors for the keys the converter pops that this repro doesn't care aboutdefpop(self, key, *default):
ifkeyinself:
returnsuper().pop(key)
ifdefault:
returndefault[0]
returntorch.zeros(21504, RANK) if".lora_B."inkeyelsetorch.zeros(RANK, 8)
lora_A=torch.randn(RANK, INNER)
lora_B=torch.randn(2*INNER, RANK) # rows: (shift, scale)sd=FillingDict({
"final_layer.adaLN_modulation.1.lora_A.weight": lora_A.clone(),
"final_layer.adaLN_modulation.1.lora_B.weight": lora_B.clone(),
"double_blocks.0.img_attn.norm.query_norm.scale": torch.ones(1),
})
out=_convert_bfl_flux_control_lora_to_diffusers(sd)
A=out["transformer.norm_out.linear.lora_A.weight"]
B=out["transformer.norm_out.linear.lora_B.weight"]
shift, scale= (lora_B @ lora_A).chunk(2, dim=0)
expected=torch.cat([scale, shift])
print(torch.allclose(B @ A, expected, atol=1e-4), (B @ A-expected).abs().max().item())
print(torch.allclose(B @ lora_A, expected, atol=1e-4)) # lora_A left as is
Logs
False 39.052650451660156
True
The HunyuanVideo converter gives the same result (max error 17.05 in the equivalent check).
System Info
diffusers main @ 43e3002, torch 2.15.0, Python 3.12, macOS (CPU).
I have a fix with tests for both converters (swap only lora_B; one test per converter in the style of test_lora_bfl_final_layer_modulation, both fail on main) and can open a PR if you'd like. It may change the expected slices of the slow Flux Control LoRA integration tests, which I can't regenerate without a GPU.
Independent confirmation on the shipped release — diffusers 0.43.0, torch 2.15.0, CPU, Linux: the reporter's repro runs exactly as written. The converted delta does not match the expected (scale, shift) result (False, max diff 39.0527, the same value in the report), the converted lora_A no longer equals the original, and leaving lora_A unchanged makes B @ A match the expected result exactly (True). So the fault is precisely the lora_A half-swap in _convert_bfl_flux_control_lora_to_diffusers: lora_B's rows are the layer's outputs and need the swap, lora_A's halves are rank components and do not. Happy to test a fix PR against this repro when one opens.
Activity
sayakpaul commentedon Nov 16, 2026
Feel free to open a PR. Thanks for reporting!
1OF1Research commentedon Nov 16, 2026
Independent confirmation on the shipped release — diffusers 0.43.0, torch 2.15.0, CPU, Linux: the reporter's repro runs exactly as written. The converted delta does not match the expected (scale, shift) result (
False, max diff 39.0527, the same value in the report), the convertedlora_Ano longer equals the original, and leavinglora_Aunchanged makesB @ Amatch the expected result exactly (True). So the fault is precisely thelora_Ahalf-swap in_convert_bfl_flux_control_lora_to_diffusers:lora_B's rows are the layer's outputs and need the swap,lora_A's halves are rank components and do not. Happy to test a fix PR against this repro when one opens.