tensorflow / tensorflow/probability

[Feature request] Make tfp.optimizer.lbfgs_minimize support a list of tensors as starting points.

Open
#1,137 0 comments 3 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Jupyter Notebook
Stars
4.4k
Forks
1.1k
PR merge metrics
No merged PRs in 30d

Description

I feel it is pretty confusing, that the tfp.optimizer.lbfgs_minimize optimizer and the bfgs pendant can only deal with a single real tensor as starting point, while most complex models give you a list of tensors when queried for trainable_variables. I feel it should be consistent with the rest of Tensorflow, since not every model has similar shaped variables, that can be put into a single tensor.

This gets even more confusing, when you take into account, that tfp.math.value_and_gradient() handles lists just fine, be it with a starred parameter function call.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the tfp.optimizer.lbfgs_minimize entry point and compare it with the BFGS counterpart. Review how tfp.math.value_and_gradient handles lists, then define completion as both optimizers accepting lists of differently shaped starting-point tensors consistently.

Written by the indexing model from the issue text.

Assessment

Tech stack
machine-learning
Domain
machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.