google-deepmind / google-deepmind/optax
Add SPSA optimization method
- Dominant language
- Python
- Stars
- 2.3k
- Forks
- 369
- Avg merge
- 10h 15m
- Merged PRs (30d)
- 7
Description
The Simultaneous Perturbation Stochastic Approximation (SPSA) optimisation method is a faster optimisation method.
> If the number of terms being optimized is p, then the finite-difference method takes 2p measurements of the objective function at each iteration (to form one gradient approximation), while SPSA takes only two measurements
It is also naturally suited for noisy measurements. Thus, it will be useful when simulating noisy systems.
The theory (and implementation) for SPSA is:
* Described in James C. Spall, “[An Overview of the Simultaneous Perturbation Method for Efficient Optimization](https://www.jhuapl.edu/SPSA/PDF-SPSA/Spall_An_Overview.PDF)”, 1998;
Furthermore, it is implemented:
* In the [`noisyopt` package](https://noisyopt.readthedocs.io/en/latest/), specifically see [the source code here](https://github.com/andim/noisyopt/blob/master/noisyopt/main.py#L264);
* In Qiskit, [see the SPSA optimizer documentation](https://qiskit.org/documentation/stubs/qiskit.algorithms.optimizers.SPSA.html#qiskit.algorithms.optimizers.SPSA).
More information:
https://www.jhuapl.edu/SPSA/
Contributor guide
Assessment
This issue has not been assessed yet.