tensorflow / tensorflow/java

Framework: Move Ops parameter to call method where possible

Open
#202 25 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Java
Stars
928
Forks
227
PR merge metrics
No merged PRs in 30d

Description

I'd like to move the Ops parameters of framework classes to the call method, where possible. This is primarily for Kotlin interop, but has a few other benefits as well. It won't be possible for stateful classes (Metrics, Optimizers), but should be possible for most, as far as I can tell (Initializers, Activations, Losses). I'm going to use Losses as a standin for all 3 for my examples.

  1. Kotlin interop w/ @FunctionInterface. If we do this, we can then define new losses like Loss{ tf, x -> tf.stuff(x) } or pass lambdas to methods that take losses. This is very nice for layers, where we might have something Keras-like Layer(activation=ReLU()) but want to replace it with something custom.
  2. Re-use of objects. Currently, losses create any subsequent calls in the same scope as their first call. That means if it's called inside a sub-scope, including device ones, it ignores the scope. This is somewhat expected, but not ideal. It also causes further issues if we use Ops for (eager) tensor lifetime management, which has been suggested and is something I'd like to do (it's easy enough to make a long-lived copy of the initial scope, but then the tensors created in the call methods live forever, and the framework classes need to be closable).
  3. Passing configs, like Keras. In Keras, if you have a activation or loss that requires some parameters, you can pass it to a layer like Layer(activation=LeakyReLU(alpha=0.3)). I expect this will be common with our API, as well. Currently, this runs into the above issue w/ scoping, and prevents you from passing activations (or losses) from scopes that don't have an Ops instance available.

I'd look at having the stateful classes take Ops in call as well, and only using the constructor ops for initializing state. This works better with scoping and lifetimes as mentioned above.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the framework classes for Initializers, Activations, and Losses, then compare the stateful Metrics and Optimizers classes. Determine which classes can receive Ops in call rather than construction, while retaining constructor Ops only for state initialization where needed. Done means the applicable APIs support Kotlin-friendly calls and preserve the intended scoping and lifetime behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, kotlin
Domain
api, backend-api-design
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.