pymc-devs / pymc-devs/pytensor
Support for `hessp` in `tensor.optimize.minimize`
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 644
- Forks
- 208
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 16
Description
Description
Currently, we don't allow the hessp argument in minimize, which sucks.
We don't allow it because currently we only have a single inner function that computes everything the user requests -- value, grad, and hess. To make this not wasteful, we wrap the inner function in a LRU cache wrapper. Calls for hess will always end up as cache hits, because of how the algorithms in scipy work -- they call value and grad (which is allowed to be fused), then evaluate the hessian at the same point as well.
I experimented with putting the hessp into the LRU1 cache as well, but it never results in cache hits. This is because there will be a call to value_and_grad at some point, then a second call to hessp at the same point, but with different values of p. If p is included in the cache hit detection, it always misses. So the strategy doesn't work for hessp.
@ricardoV94 says we are allowed to have two inner functions. So if the user asks for hessp, we need to compile a second inner function to compute the hessp, and it can be used inside the perform method.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in tensor.optimize.minimize and inspect the existing inner function, LRU cache behavior, and perform method. Trace how hess is compiled and evaluated, then add the separate hessp path described in the issue and verify that minimize accepts and evaluates hessp without relying on cache hits.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100