Skip to content

[MiniMax-H3] Load the pruned ComfyUI checkpoints and add single-file support for MiniMax-H3-Image #15302

Open

Description

@guoriyue

Is your feature request related to a problem? Please describe.

ComfyUI distributes pruned MiniMax-H3 checkpoints (Comfy-Org/MiniMax-H3, Comfy-Org/MiniMax-H3-Image). They cannot be loaded in diffusers today:

  • The pruned files replace the timestep MLP and the wide AdaLN projections (about 40% of the parameters) with the activated timestep curve sampled on a 1025-point grid (adaln_t_table) and 8-input AdaLN projections. MiniMaxH3Transformer3DModel has no module that matches these weights, and from_single_file refuses adaln_t_table.
  • MiniMaxH3ImageTransformer2DModel has no single-file support at all (no FromOriginalModelMixin), and the ComfyUI file stores the PDD heads as a stacked bank (head 0 plus per-step offsets) rather than as pdd_proj_out.{i}.

Describe the solution you'd like.

  1. An opt-in config field (adaln_curve_grid) on both transformers. When set, time_embedder is a lookup of the sampled curve, linearly interpolated as ComfyUI does, and the AdaLN projections skip their SiLU. Stock configs keep their current behaviour.
  2. from_single_file for both transformers: map adaln_t_table to the table, unstack the PDD head bank for the image model, and refuse quantized ComfyUI files with a message naming the bf16 file.
transformer = MiniMaxH3ImageTransformer2DModel.from_single_file(
    "https://huggingface.co/Comfy-Org/MiniMax-H3-Image/blob/main/diffusion_models/minimax_h3_image_pruned_bf16.safetensors",
    dtype=torch.bfloat16,
)

Design question for maintainers

The repository config on the Hub describes the unpruned model, and no Hub repo carries a pruned diffusers config that a model type could point at. So the pruned shape (adaln_curve_grid, time_embed_dim) has to be read off the checkpoint itself. #15299 does this with a new optional key in the single-file registry, config_from_checkpoint_fn. Is a new registry key acceptable here, or would you prefer another route (for example a separate model type with its own config)?

Additional context.

Implementation proposed in #15299.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions