clab / clab/dynet

minibatch on GPU

Open
#1,417 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
3.4k
Forks
701
PR merge metrics
No merged PRs in 30d

Description

Followed the [instruction](http://dynet.readthedocs.io/en/latest/minibatch.html#automatic-mini-batching), and cooked up minibatches with 5000 words (roughly 250 CGs will be gathered in each mini-batch before the loss is computed and weights are updated at the end of the minibatch). Then switched on --dynet-autobatch 1 --dynet-gpus 1 and run it on GPU. Without these flags, it runs just fine on CPU. But on GPU, it is several times slower. If I grow the size of the minibatch, it will run out of memory on GPU. Is there something silly that I am doing?

[For the record, the model is the Graph-based [BiLSTM parser](https://github.com/elikip/bist-parser). Also for the record, if I switch on the external embeddings (which concatenates the input with external embeddings), then it becomes immediately impractical to even run it on smaller batch sizes on GPU. Runs out of memory on GPU. The GPU has 8GB of memory.]

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the linked BiLSTM parser workload with --dynet-autobatch 1 and --dynet-gpus 1, comparing it with the CPU run and recording GPU memory use. Check the automatic mini-batching instruction and the parser's external-embeddings configuration first. Done means the GPU minibatch performance and memory behavior are explained and a concrete fix or documented limitation is identified.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
machine-learning, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.