clab / clab/dynet

bug(autobatch): autobatching slows training by at least an order of magnitude

Open
#1,222 7 comments 0 reactions 0 assignees View on GitHub
moderate bug
Dominant language
C++
Stars
3.4k
Forks
701
PR merge metrics
No merged PRs in 30d

Description

I have this model:
> Parameters:
> Forward LSTM 372 x 31 = 11532
> Backward LSTM 62 x 31 = 1922
> MLP 186 x 31 = 5766
> Embeddings 2 x 31 = 62
> Concatted MLPs 2 x 138 x 138 = 38088
> Final MLPs 31 x 138 x 5 = 21390

And I am training on CPU.

I batch my training data into batches of 50, and run the "backward" method on an `esum` of errors.

When I add `--dynet-autobatch 1` the model takes at least an order of magnitude more time to run training.
Is this a bug? Am I doing something wrong for autobatching?

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the reported CPU training setup with batches of 50, the `backward` call on an `esum` of errors, and `--dynet-autobatch 1`; compare its timing with autobatching disabled. Determine whether the order-of-magnitude slowdown is reproducible and whether it reflects a bug or an incorrect usage pattern.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.