Component override system doesn't fully support GGUF or video pipeline architectures

Problem

The component loading and override system in the Load Pipeline Component node (and modular_diffusion_nodes_library/artifact_utils/component_artifact.py) has several gaps that make it hard to use GGUF quantized models, and hard to configure video generation pipelines at all. Four distinct gaps, all in the same area:

Loading a GGUF file requires a separate config source, and that requirement is undiscoverable

GGUF files are self-contained single files, but loading one also requires a config_source pointing at a separate config.json (the model architecture definition). That config typically lives deep in the HuggingFace cache — e.g. ~/.cache/huggingface/hub/models--<org>--<model>/.../transformer/config.json — and it has to match the exact architecture the GGUF was quantized from.

Someone who downloads a GGUF from HuggingFace won't know they need this, won't know where to find it, and won't know which repo's config matches their file. The failure messages don't help:

• The first attempt fails with Cannot copy out of meta tensor, which never mentions config_source at all.
• Earlier in the flow, a failed config lookup reports model_type='ltx-video-0.9.7' config not found — pointing at the wrong model entirely, because the GGUF is mis-detected as an older architecture.

Desired outcome: a user with just a GGUF file can load it, or is told clearly what's missing and where to get it.

Only "Transformer" is offered as a component slot for GGUF sources

When the source is a single-file GGUF, the Load Pipeline Component node offers "Transformer" as the only component slot. Video pipelines such as LTX2 have further components a user may reasonably want to override or load from a quantized source — connectors, VAE variants, text encoders — and there's currently no way to target any of them through the UI when the source is a GGUF.

GGUF checkpoint key conversion fails silently

In _materialize_single_file, the checkpoint is run through diffusers' key conversion functions. Those were written for original-format safetensors, and GGUF converters sometimes use different key naming conventions. When keys go unmatched, the affected parameters stay on the meta device and the model crashes with Cannot copy out of meta tensor.

There is no validation step, no diagnostic logging of partial key matches, and no hook for patching known mapping gaps — so a naming divergence surfaces only as an opaque crash much later. This is not specific to one model: any GGUF whose key names diverge from what the corresponding convert_*_to_diffusers function expects hits it.

Video pipelines need more than a single VAE slot

The pipeline builder's component override system assumes one "VAE" slot. Video generation pipelines like LTX2 have multiple decode/encode components — a video VAE, an audio VAE, and video and audio embedding connectors. These are distinct components with separate weights, but the override UI doesn't expose them as individually configurable slots, so they can't be swapped, quantized, or configured independently through the standard pipeline builder workflow.

Origin

Reported by Joshua Bretag. Surface: Load Pipeline Component node and the pipeline builder's component override system in this library, exercised against GGUF quantized models and LTX2 video pipelines.

Impact

GGUF quantized models are effectively unusable without prior knowledge of the config_source requirement and the HuggingFace cache layout, and even then the silent key-conversion gap can make a given file fail with no diagnosable cause. Video pipelines can't have their decode/encode components configured independently at all. Quantized models matter for local diffusion workflows on constrained hardware, so this blocks a fairly central use case.

Repro / evidence

Loading a single-file GGUF through the Load Pipeline Component node:

1. Provide only the GGUF file as the source → fails with Cannot copy out of meta tensor, with no indication a config is needed.
2. Config lookup earlier in the flow reports model_type='ltx-video-0.9.7' config not found, naming an architecture unrelated to the GGUF actually being loaded.
3. Supplying a matching config.json from the base repo's HuggingFace cache directory gets past the load, provided the config matches the architecture the file was quantized from.

Possible direction

Options from the report, not a prescribed fix:

• Auto-resolve the config from the GGUF's own metadata or from the base repo, rather than requiring the user to locate one — or, failing that, surface a message that explains what config_source is and where to find it.
• Expose the additional components of video pipelines (connectors, VAE variants, text encoders; video VAE, audio VAE, video and audio embedding connectors) as individually configurable override slots.
• Add validation and diagnostic logging around GGUF key conversion so unmatched keys are reported at conversion time, plus a hook for patching known mapping gaps.

Related

• The opaque Cannot copy out of meta tensor message is tracked more generally in #17; the key-conversion gap above is one concrete source of it.
• griptape-ai/griptape-nodes-library-advanced-media#2 requests GGUF support in the Diffusion Pipeline Builder.