Improve robustness of weight optimisers
- Dominant language
- Python
- Stars
- 43
- Forks
- 6
- Avg merge
- 3d 22h
- Merged PRs (30d)
- 10
Description
We currently have two weights' solvers: SBQ and simplex. The former allows for arbitrary weights; the latter for simplex weights only.
For the Stein kernel, arbitrary weights give a degenerate solution, i.e. $w^* = \mathbf{0}$, since the kernel mean embedding $\int_\mathcal{X} \kappa(\cdot, \mathbf{x}) \mathrm{d}\mathbb{P}$ is zero under the target measure $\mathbb{P}$. The optimal weights require a sum-to-one condition (though not a non-negative one), which gives
$$
\mathbf{w}^* = \frac{\sum_j \mathbf{K}_{:, j}^{-1}}{\sum_i \sum_j \mathbf{K}\_{i, j}^{-1}},
$$
where $\mathbf{K}$ is the Stein kernel Gram matrix.
I would like to add some new weights' solvers to deal with, _inter alia_:
- The Stein kernel solution above.
- Methods to deal with the matrix inversion and ill-conditioned linear systems, cf. negative weights bug [#101](https://github.com/gchq/coreax/issues/101).
- Methods to more efficiently compute/estimate weights, e.g. Section 3.4 of [_Fully symmetric kernel quadrature_ (Karvonen and Särkkä)](https://arxiv.org/pdf/1703.06359.pdf).
Contributor guide
Research direction
Start by locating the existing SBQ and simplex weight-solver implementations and their tests, then read issue #101 for the negative-weights failure mode. Compare the current solver assumptions with the Stein-kernel sum-to-one formula and the proposed efficient-weight methods. Done should include clearly scoped solver additions and coverage for ill-conditioned systems and the Stein-kernel case.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100