dmlc / dmlc/MXNet.jl

Dynamic dispatching in optimizers

Open
#44 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
No language data
Stars
369
Forks
63
PR merge metrics
No merged PRs in 30d

Description

I could be wrong in this, but looking at the code for [sgd](https://github.com/dmlc/MXNet.jl/blob/master/src/optimizers/sgd.jl) and [adam](https://github.com/dmlc/MXNet.jl/blob/master/src/optimizers/adam.jl) it looks like there is dynamic function dispatch going on because the types for optimizer options are not known at compile time.

For example for sgd in `update`:

``` Julia
lr = get_learning_rate(self.opts.lr_scheduler, self.state)
```

If I understand Julia correctly, then `lr_scheduler` is a boxed value in an `Any` block and thus the appropriate `get_learning_rate` variant needs to be looked up at runtime, which can have performance implications.

What do you think?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.