pymc-devs / pymc-devs/pytensor

Support for `hessp` in `tensor.optimize.minimize`

Open
#1,472 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement feature request help wanted SciPy compatibility
Dominant language
Python
Stars
644
Forks
208
Avg merge
2d 14h
Merged PRs (30d)
16

Description

Description

Currently, we don't allow the hessp argument in minimize, which sucks.

We don't allow it because currently we only have a single inner function that computes everything the user requests -- value, grad, and hess. To make this not wasteful, we wrap the inner function in a LRU cache wrapper. Calls for hess will always end up as cache hits, because of how the algorithms in scipy work -- they call value and grad (which is allowed to be fused), then evaluate the hessian at the same point as well.

I experimented with putting the hessp into the LRU1 cache as well, but it never results in cache hits. This is because there will be a call to value_and_grad at some point, then a second call to hessp at the same point, but with different values of p. If p is included in the cache hit detection, it always misses. So the strategy doesn't work for hessp.

@ricardoV94 says we are allowed to have two inner functions. So if the user asks for hessp, we need to compile a second inner function to compute the hessp, and it can be used inside the perform method.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in tensor.optimize.minimize and inspect the existing inner function, LRU cache behavior, and perform method. Trace how hess is compiled and evaluated, then add the separate hessp path described in the issue and verify that minimize accepts and evaluates hessp without relying on cache hits.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.