Feature: LFM2-MoE and LFM2-VL follow-on families (scoping)

Request type

New model or checkpoint support

Problem and use case

Follow-up to #928, where dense LFM2 support landed (#937) and maintainers offered collaboration on the remaining variants. Two released LiquidAI lines are still unsupported: LFM2-8B-A1B (Lfm2MoeForCausalLM, model_type lfm2_moe) and LFM2-VL 450M/1.6B (Lfm2VlForConditionalGeneration, model_type lfm2_vl). Both reuse the dense backbone semantics that are now native.

Proposed outcome

Two new model-owned families, phased.

Phase 1, lfm2_moe: the checkpoint is the known hybrid backbone (24 layers, 17 conv + 7 full attention, hidden 2048) with the per-layer SwiGLU replaced by sparse MoE on layers 2 to 23: 32 experts, top-4 routing, moe_intermediate_size 1792, first 2 layers dense. Deltas from the dense family stay contained to the FFN block (router + expert dispatch in the builder, expert weight layout in the mapper, top-k dispatch in the runtime) while conv, attention, and state handling carry over. Strategy key lfm2_moe_hybrid_conv_attention, own DSO, E2E vs hf_transformers at a pinned revision.

Phase 2, lfm2_vl: SigLIP2 vision tower (1152 hidden, 26 layers, patch 16) plus projector feeding the unmodified lfm2 text backbone via the embed_input contract, following the existing VL family patterns (eagle_vlm, internvl). Needs agreement on image preprocessing strategy and on whether the text side shares builder modules or carries family-local copies per the ownership rules.

Model and target details

LiquidAI/LFM2-8B-A1B (bf16; expected to fit a single L4/A10G class GPU, to be confirmed during bring-up), then LiquidAI/LFM2-VL-450M and LFM2-VL-1.6B. task_strategy text_generation_causal and vision_language_generation respectively.

Alternatives considered

Extending the dense lfm2 family in place: rejected, the repo contract keeps distinct checkpoint layouts and request contracts in their own model-owned families.

Additional context

I'd like to implement both, starting with lfm2_moe. Three questions before I start: (1) any in-flight internal work on either variant I should not duplicate, (2) preferred expert-weight layout for TRT (fused expert tensors vs per-expert constants), (3) for VL, the preferred preprocessor_type given SigLIP2's resizing behavior. Build and parity evidence on rented hardware, as with the dense line.

Submission checks

• I searched open and closed issues and found no duplicate request.
• I removed secrets, private/internal evidence, personal paths, and restricted artifacts.

Start by reviewing dense LFM2 support from #937, then compare the existing eagle_vlm and internvl family patterns. Trace the builder, mapper, runtime, and embed_input contracts named in the request, and confirm ownership and preprocessing decisions with maintainers. Done means an agreed phase plan plus implementation and pinned hf_transformers E2E parity evidence for each supported family.