minibatch on GPU
- Dominant language
- C++
- Stars
- 3.4k
- Forks
- 701
- PR merge metrics
- No merged PRs in 30d
Description
Followed the [instruction](http://dynet.readthedocs.io/en/latest/minibatch.html#automatic-mini-batching), and cooked up minibatches with 5000 words (roughly 250 CGs will be gathered in each mini-batch before the loss is computed and weights are updated at the end of the minibatch). Then switched on --dynet-autobatch 1 --dynet-gpus 1 and run it on GPU. Without these flags, it runs just fine on CPU. But on GPU, it is several times slower. If I grow the size of the minibatch, it will run out of memory on GPU. Is there something silly that I am doing?
[For the record, the model is the Graph-based [BiLSTM parser](https://github.com/elikip/bist-parser). Also for the record, if I switch on the external embeddings (which concatenates the input with external embeddings), then it becomes immediately impractical to even run it on smaller batch sizes on GPU. Runs out of memory on GPU. The GPU has 8GB of memory.]
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the linked BiLSTM parser workload with --dynet-autobatch 1 and --dynet-gpus 1, comparing it with the CPU run and recording GPU memory use. Check the automatic mini-batching instruction and the parser's external-embeddings configuration first. Done means the GPU minibatch performance and memory behavior are explained and a concrete fix or documented limitation is identified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- machine-learning, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100