Dynamic dispatching in optimizers
- Dominant language
- No language data
- Stars
- 369
- Forks
- 63
- PR merge metrics
- No merged PRs in 30d
Description
I could be wrong in this, but looking at the code for [sgd](https://github.com/dmlc/MXNet.jl/blob/master/src/optimizers/sgd.jl) and [adam](https://github.com/dmlc/MXNet.jl/blob/master/src/optimizers/adam.jl) it looks like there is dynamic function dispatch going on because the types for optimizer options are not known at compile time.
For example for sgd in `update`:
``` Julia
lr = get_learning_rate(self.opts.lr_scheduler, self.state)
```
If I understand Julia correctly, then `lr_scheduler` is a boxed value in an `Any` block and thus the appropriate `get_learning_rate` variant needs to be looked up at runtime, which can have performance implications.
What do you think?
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.