gchq / gchq/coreax

Improve robustness of weight optimisers

Open
#134 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
43
Forks
6
Avg merge
3d 22h
Merged PRs (30d)
10

Description

We currently have two weights' solvers: SBQ and simplex. The former allows for arbitrary weights; the latter for simplex weights only.

For the Stein kernel, arbitrary weights give a degenerate solution, i.e. $w^* = \mathbf{0}$, since the kernel mean embedding $\int_\mathcal{X} \kappa(\cdot, \mathbf{x}) \mathrm{d}\mathbb{P}$ is zero under the target measure $\mathbb{P}$. The optimal weights require a sum-to-one condition (though not a non-negative one), which gives

$$
\mathbf{w}^* = \frac{\sum_j \mathbf{K}_{:, j}^{-1}}{\sum_i \sum_j \mathbf{K}\_{i, j}^{-1}},
$$

where $\mathbf{K}$ is the Stein kernel Gram matrix.

I would like to add some new weights' solvers to deal with, _inter alia_:

- The Stein kernel solution above.
- Methods to deal with the matrix inversion and ill-conditioned linear systems, cf. negative weights bug [#101](https://github.com/gchq/coreax/issues/101).
- Methods to more efficiently compute/estimate weights, e.g. Section 3.4 of [_Fully symmetric kernel quadrature_ (Karvonen and Särkkä)](https://arxiv.org/pdf/1703.06359.pdf).

Contributor guide

Open the contributing guide

Research direction

Start by locating the existing SBQ and simplex weight-solver implementations and their tests, then read issue #101 for the negative-weights failure mode. Compare the current solver assumptions with the Stein-kernel sum-to-one formula and the proposed efficient-weight methods. Done should include clearly scoped solver additions and coverage for ill-conditioned systems and the Stein-kernel case.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.