bytedance / bytedance/UniVR

Inference BOS fallback crashes when constructing the prefix tensor

Open
#5 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
32
Forks
2
PR merge metrics
No merged PRs in 30d

Description

### Affected revision and environment

`main` at `3d1965cac5529b72c9dab9af7a3f456c5bbd7172`; reproduced with PyTorch `2.12.0.dev20260408+cu128`. The constructor behavior is also invalid in supported stable PyTorch releases.

### Reproduction

The inference fallback used when a tokenized prompt does not begin with BOS is equivalent to:

```python
import torch
torch.Tensor([[203]], dtype=torch.long)
```

This raises:

```text
TypeError: new() received an invalid combination of arguments - got (list, dtype=torch.dtype)
```

The same call appears twice in `UniVR_SFT/inference.py` and twice in `UniVR_SFT/inference_and_evaluate.py`.

### Expected behavior

The fallback should prepend a one-element BOS tensor with the same dtype and device as `input_ids`.

### Actual behavior

`torch.Tensor` is the legacy class constructor and does not accept `dtype` with list data, so inference stops before generation whenever this fallback branch is reached.

### Impact

Custom prompt templates or tokenizers that do not already emit the configured BOS token cannot use either inference entry point.

### Suggested fix

Use `torch.tensor(..., dtype=input_ids.dtype, device=input_ids.device)` at all four source sites and add a regression check for the constructor form.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by inspecting the two occurrences in UniVR_SFT/inference.py and the two in UniVR_SFT/inference_and_evaluate.py, then reproduce the fallback constructor failure with the issue's PyTorch example. Update all four source sites so the BOS tensor preserves input_ids dtype and device, and add a regression check confirming inference can enter this fallback branch.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.