ByteDance-Seed / ByteDance-Seed/Bagel

Error when running evaluation script: Cannot copy out of meta tensor

Open
#25 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
6.2k
Forks
545
PR merge metrics
No merged PRs in 30d

Description

When running the evaluation script for the BAGEL model, I encountered the following error:

```
NotImplementedError: Cannot copy out of meta tensor; no data!
```

Steps to reproduce:
1. Set up environment variables (OPENAI_API_KEY, GPUS, ARNOLD_* variables, etc.)
2. Run the evaluation script:

```bash
bash scripts/eval/eval_vlm.sh \
$BASE_RESUME_DIR \
$MODEL_NAME \
$OUTPUT_DIR \
--model-path $model_path
```

Full error trace:
```
_IncompatibleKeys(missing_keys=[], unexpected_keys=['latent_pos_embed.pos_embed', 'llm2vae.bias', 'llm2vae.weight', 'time_embedder.mlp.0.bias', 'time_embedder.mlp.0.weight', 'time_embedder.mlp.2.bias', 'time_embedder.mlp.2.weight', 'vae2llm.bias', 'vae2llm.weight'])
Traceback (most recent call last):
File "/usr/local/conda/envs/dx-bagel/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/local/conda/envs/dx-bagel/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File ".../eval/vlm/eval/mme/eval.py", line 41, in
model, tokenizer, new_token_ids = load_model_and_tokenizer(args)
File ".../eval/vlm/utils.py", line 69, in load_model_and_tokenizer
model = model.cuda().eval()
File "/usr/local/conda/envs/dx-bagel/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3122, in cuda
return super().cuda(*args, **kwargs)
File "/usr/local/conda/envs/dx-bagel/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1050, in cuda
return self._apply(lambda t: t.cuda(device))
File "/usr/local/conda/envs/dx-bagel/lib/python3.10/site-packages/torch/nn/modules/module.py", line 900, in _apply
module._apply(fn)
File "/usr/local/conda/envs/dx-bagel/lib/python3.10/site-packages/torch/nn/modules/module.py", line 900, in _apply
module._apply(fn)
File "/usr/local/conda/envs/dx-bagel/lib/python3.10/site-packages/torch/nn/modules/module.py", line 900, in _apply
module._apply(fn)
File "/usr/local/conda/envs/dx-bagel/lib/python3.10/site-packages/torch/nn/modules/module.py", line 927, in _apply
param_applied = fn(param)
File "/usr/local/conda/envs/dx-bagel/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1050, in
return self._apply(lambda t: t.cuda(device))
NotImplementedError: Cannot copy out of meta tensor; no data!
```

Environment:
- Python version: 3.10
- CUDA version: [Please specify if known]
- BAGEL version: [Please specify if known]

Any help in resolving this issue would be appreciated. Thank you!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the failure with scripts/eval/eval_vlm.sh and the reported environment variables. Read eval/vlm/eval/mme/eval.py at model loading and eval/vlm/utils.py line 69, where model.cuda().eval() fails. Done means the BAGEL evaluation model loads onto CUDA and proceeds without the meta-tensor error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.