probml / probml/dynamax

Support missing (`nan`) emissions in `LinearGaussianConjugateSSM.fit_blocked_gibbs`

Open
#399 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1k
Forks
114
Avg merge
19h 14m
Merged PRs (30d)
1

Description

Hi,

First off, thanks for building this awesome package. Well done!
I work in the hospital and I am interested in fitting a Kalman filter on sparsely measured blood levels where most observations are missing. Am I correct in assuming that missing values are currently not supported by LinearGaussianConjugateSSM?
If so, in theory this is not so difficult to implement in the fit_blocked_gibbs if an entire emission is missing, I think.

  1. Update forward filtering backwards sampling algorithm with the missing observations integrated out.
  2. Update the parameter sampling. Integrating out the missing observations would amount to removing the control-emission pairs from the emission summary statistics where the emission value is missing. More complex would be when part of the emission is missing (giving marginal emissions). That probably requires a generalization of the normal-inverse Wishart distribution. See also this paper: Missing observation analysis for matrix-variate time series data.

If you're interested, I can give the complete missing observation a try. Would you accept such a pull request?

Kind regards,

Hylke

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at LinearGaussianConjugateSSM.fit_blocked_gibbs and trace the forward filtering backward sampling and parameter-sampling paths. Done would mean supporting entirely missing emissions by integrating them out and excluding their control-emission pairs from emission summary statistics; partially missing emissions are described as a more complex follow-up.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.