ByteDance-Seed / ByteDance-Seed/Bagel
Error when running evaluation script: Cannot copy out of meta tensor
- Dominant language
- Python
- Stars
- 6.2k
- Forks
- 545
- PR merge metrics
- No merged PRs in 30d
Description
When running the evaluation script for the BAGEL model, I encountered the following error:
```
NotImplementedError: Cannot copy out of meta tensor; no data!
```
Steps to reproduce:
1. Set up environment variables (OPENAI_API_KEY, GPUS, ARNOLD_* variables, etc.)
2. Run the evaluation script:
```bash
bash scripts/eval/eval_vlm.sh \
$BASE_RESUME_DIR \
$MODEL_NAME \
$OUTPUT_DIR \
--model-path $model_path
```
Full error trace:
```
_IncompatibleKeys(missing_keys=[], unexpected_keys=['latent_pos_embed.pos_embed', 'llm2vae.bias', 'llm2vae.weight', 'time_embedder.mlp.0.bias', 'time_embedder.mlp.0.weight', 'time_embedder.mlp.2.bias', 'time_embedder.mlp.2.weight', 'vae2llm.bias', 'vae2llm.weight'])
Traceback (most recent call last):
File "/usr/local/conda/envs/dx-bagel/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/local/conda/envs/dx-bagel/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File ".../eval/vlm/eval/mme/eval.py", line 41, in
model, tokenizer, new_token_ids = load_model_and_tokenizer(args)
File ".../eval/vlm/utils.py", line 69, in load_model_and_tokenizer
model = model.cuda().eval()
File "/usr/local/conda/envs/dx-bagel/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3122, in cuda
return super().cuda(*args, **kwargs)
File "/usr/local/conda/envs/dx-bagel/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1050, in cuda
return self._apply(lambda t: t.cuda(device))
File "/usr/local/conda/envs/dx-bagel/lib/python3.10/site-packages/torch/nn/modules/module.py", line 900, in _apply
module._apply(fn)
File "/usr/local/conda/envs/dx-bagel/lib/python3.10/site-packages/torch/nn/modules/module.py", line 900, in _apply
module._apply(fn)
File "/usr/local/conda/envs/dx-bagel/lib/python3.10/site-packages/torch/nn/modules/module.py", line 900, in _apply
module._apply(fn)
File "/usr/local/conda/envs/dx-bagel/lib/python3.10/site-packages/torch/nn/modules/module.py", line 927, in _apply
param_applied = fn(param)
File "/usr/local/conda/envs/dx-bagel/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1050, in
return self._apply(lambda t: t.cuda(device))
NotImplementedError: Cannot copy out of meta tensor; no data!
```
Environment:
- Python version: 3.10
- CUDA version: [Please specify if known]
- BAGEL version: [Please specify if known]
Any help in resolving this issue would be appreciated. Thank you!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the failure with scripts/eval/eval_vlm.sh and the reported environment variables. Read eval/vlm/eval/mme/eval.py at model loading and eval/vlm/utils.py line 69, where model.cuda().eval() fails. Done means the BAGEL evaluation model loads onto CUDA and proceeds without the meta-tensor error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100