Inference BOS fallback crashes when constructing the prefix tensor
- Dominant language
- Python
- Stars
- 32
- Forks
- 2
- PR merge metrics
- No merged PRs in 30d
Description
### Affected revision and environment
`main` at `3d1965cac5529b72c9dab9af7a3f456c5bbd7172`; reproduced with PyTorch `2.12.0.dev20260408+cu128`. The constructor behavior is also invalid in supported stable PyTorch releases.
### Reproduction
The inference fallback used when a tokenized prompt does not begin with BOS is equivalent to:
```python
import torch
torch.Tensor([[203]], dtype=torch.long)
```
This raises:
```text
TypeError: new() received an invalid combination of arguments - got (list, dtype=torch.dtype)
```
The same call appears twice in `UniVR_SFT/inference.py` and twice in `UniVR_SFT/inference_and_evaluate.py`.
### Expected behavior
The fallback should prepend a one-element BOS tensor with the same dtype and device as `input_ids`.
### Actual behavior
`torch.Tensor` is the legacy class constructor and does not accept `dtype` with list data, so inference stops before generation whenever this fallback branch is reached.
### Impact
Custom prompt templates or tokenizers that do not already emit the configured BOS token cannot use either inference entry point.
### Suggested fix
Use `torch.tensor(..., dtype=input_ids.dtype, device=input_ids.device)` at all four source sites and add a regression check for the constructor form.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by inspecting the two occurrences in UniVR_SFT/inference.py and the two in UniVR_SFT/inference_and_evaluate.py, then reproduce the fallback constructor failure with the issue's PyTorch example. Update all four source sites so the BOS tensor preserves input_ids dtype and device, and add a regression check confirming inference can enter this fallback branch.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100