mudler / mudler/vllm.cpp

port(LORA-RUNTIME): ExpandPackedLora refuses a packed group with an unadapted member, which is partial LoRA on GatedDeltaNet

Open
#2,721 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
423
Forks
53
Avg merge
20h 26m
Merged PRs (30d)
310

Description

Row: LORA-RUNTIME (.agents/engine-matrix.md:256, re-resolved by line number before filing)

Found by wave PORTQ-6 (#2718) re-deriving PORT-NOW entry [205], upstream c2e7242ab7 vllm#47640, of 5559679229..e126687a9a. Nothing was executed. This is a static reading of source.

What upstream does

vllm/lora/layers/column_parallel_linear.py widens expand_packed_lora's signature to list[torch.Tensor | None] and inserts a if b_i is None: guard at the top of the loop. An unadapted packed-group member emits self.n_slices - start_idx None placeholders, advances start_idx, and continues, so the remaining slices stay aligned and are left at base weights. Before the fix the first statement of the loop body was b_rows, cu_rows, covered = b_i.shape[0], 0, 0, which dereferences None. The case is partial LoRA on a Qwen3.5/3.6 GatedDeltaNet.

The pre-fix shape is at the pin: git show 5559679229:vllm/lora/layers/column_parallel_linear.py:266-299. This fix is post-pin and is owed by the pin advance.

What is here

ExpandPackedLora is ported, and it is exactly the pre-commit shape.

// src/vllm/lora/layers.cpp:354-355
for (size_t g = 0; g < lora_b.size(); ++g) {
  const int64_t b_rows = lora_b[g].rows;

There is no emptiness guard. An absent sub-module is an empty LoRAMatinclude/vllm/lora/layers.h:66-78 gives rows = 0 and empty() as rows == 0 || cols == 0, and :79-80 documents MatList as upstream's list[torch.Tensor | None] with "an EMPTY LoRAMat standing for None". So b_rows reads 0; the inner loop at layers.cpp:357-363 adds a positive output_sizes_[i] before every comparison and can never reach cu_rows == 0; covered stays 0 and :364-369 throws std::invalid_argument.

The asymmetry is the point. The None tolerance upstream's docstring cites as already present is present here, on both of the other two paths: MergedColumnParallelLinearWithLoRA::SliceLoraB (src/vllm/lora/layers.cpp:333-343, if (lora_b[ii].empty()) continue;) and the stacking loop in BaseLinearLayerWithLoRA::SetLora (:252-263). ExpandPackedLora is the one that was never given it, and it is entered from MergedColumnParallelLinearWithLoRA::SetLora at layers.cpp:389-392 whenever lora_b.size() != n_slices_, which is the partial-group case.

Consequence: a refusal, not a crash and not silent wrongness

std::invalid_argument("LoRA: cannot split lora_b with 0 rows into N slices starting at S").

It is not user-visible today. include/vllm/lora/layers.h is included by src/vllm/lora/layers.cpp and nothing else, and the only constructors of MergedColumnParallelLinear{,VariableSlice}WithLoRA are in tests/vllm/lora/test_lora_layers.cpp. The LoRA layer family is a staged unreached slice, which the LORA-RUNTIME row already records ("the CPU brick exists but nothing reaches it", .agents/feature-matrix.md:39). The gap becomes reachable the day the seam is wired, which is the argument for fixing it in the same change rather than after.

Size

~15 product lines in src/vllm/lora/layers.cpp: an if (lora_b[g].empty()) branch that appends n_slices_ - start_idx empty LoRAMats to both lists, advances start_idx and continues. ~35 test lines in tests/vllm/lora/test_lora_layers.cpp, red-first.

Probe note

The queue text asserts "our layers.h ports expand_packed_lora". That was verified, not assumed: ExpandPackedLora returns 4 hits over src/, include/ and tests/. The secondary zero — no existing test drives an empty member through ExpandPackedLora — carries its control: test_lora_layers.cpp:290 asserts CHECK_FALSE(packed.subloras[1].has_value()); // the "None" sub-module, so the None concept is tested in that same file, just not on this path.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in src/vllm/lora/layers.cpp at ExpandPackedLora and read LoRAMat's empty representation in include/vllm/lora/layers.h. Compare the existing empty-member handling in SliceLoraB and SetLora, then add coverage in tests/vllm/lora/test_lora_layers.cpp for a partial packed group. Done means empty members are preserved as empty slices and the test passes without the invalid_argument refusal.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.