ByteDance-Seed / ByteDance-Seed/VideoWorld
Question about GPU setup for dLDM stage-2 training
- Dominant language
- Python
- Stars
- 801
- Forks
- 40
- PR merge metrics
- No merged PRs in 30d
Description
Hi, thanks for releasing VideoWorld 2!
I am trying to reproduce the dLDM training pipeline, and kept the stage-2 setting same as your released config:
- `predict2_video2world_ctrl_ldm_fsdp_2b_480p_16fps`
- `train_architecture="base"`
- `finetune_base_model=True`
- `use_cross_embedding=True`
- `batch_size=1` per GPU
- `num_frames=93`
- `video_size=(480, 832)`
- `fsdp_shard_size=8`
- 8 GPUs total
From the logs, stage 2 is training roughly:
- DiT trainable parameters: `2.20B`
- dLDM trainable parameters: `0.14B`
So this is effectively joint training of:
- full DiT
- full control branch
- dLDM
I can confirm that FSDP is enabled (`Distributed parallelism mode: fsdp`, `Using FSDP with shard size 8`), but I still run into OOM issues on 8 GPUs with 48 GB VRAM each.
Because of this, I would like to ask:
1. What exact GPU setup did you use for dLDM stage-2 training?
- number of GPUs
- VRAM per GPU
- GPU model
2. For the released training recipe, was stage 2 expected to fit on 8 x 48 GB GPUs?
3. Did you use any additional memory-saving settings beyond the released config?
For example:
- activation checkpointing changes
- different batch size / grad accumulation
- context parallelism
- different FSDP settings
- partial freezing instead of full base-model finetuning
4. If stage 2 in your internal training used a different setup than the public config, could you share the practical hardware recommendation for reproducing it?
Any clarification would be very helpful for reproduction. Thanks.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.