open-compass / open-compass/VLMEvalKit
nproc did not redistribute the data and did not accelerate inference
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.4k
- Forks
- 768
- Avg merge
- 2d 27m
- Merged PRs (30d)
- 18
Description
Hi,i am using the command
torchrun --nproc-per-node=2 run.py --config config.json to inference my own dataset.
However, after implementing this,i found that it only successfully placed one model on each GPU , but the dataset wasn't split and thus the two models were testing the same data(doing the same thing concurrently).The inference speed did not improve.
How should i fix it?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with run.py and the config.json used by torchrun --nproc-per-node=2; inspect how distributed inference initializes processes and assigns dataset work. Reproduce the command with two processes and verify that each process evaluates a distinct subset rather than the full dataset. Done means the data is partitioned across GPUs and inference throughput improves without duplicate evaluation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- distributed-systems, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100