simonsobs / simonsobs/sotodlib

Wavelet Jumps

Open
#797 3 comments 0 reactions 1 assignee View on GitHub

@skhrg is already working on this.

Since Apr 10, 2024.

preprocess
Dominant language
Python
Stars
19
Forks
23
Avg merge
1d 5h
Merged PRs (30d)
14

Description

Mostly just a looming todo item for me so that I can feel ok about putting this on the back burner and work on more pressing things.

Finding jumps with a matched filter seems to mostly work but there are some issues namely:

  1. A fourier space convolution doesn't really like the sharp discontinuity
  2. Real space convolution are slow...

But wavelets make a lot of sense here since they are good at dealing with these sorts of localized features.

A naive attempt of looking at the CWT with a gaus1 wavelet seems to work well, but CWTs are probably more expensive than we need. This paper uses a DWT dydatic scales and seems to be the basis of a lot of modern approaches (though I cannot find a python implementation of the wavelet they use). This paper is a good followup with more information about dealing with noise.

Regardless of the exact technique used I think my takeaway is that using methods like this to then get an estimate of the Lipshitz regularity would be a very powerful technique in our flagging toolkit (since that also get us glitches at the same time but in a way where we know what is what). This paper is a good overview of estimating alpha.

I can imagine two modes here that we could use:

  1. Estimate alpha at each point in the TOD and then use that to flag jumps and gltiches.
  2. Do a cheap but perhaps false positive prone selection (ie: a thresh on the gradient, this can be done in the wavelett basis as well) and then estimate alpha in each of the flagged zones to kill false positives.

Overall I think that these sorts of wavelet techniques have the potential to be the final boss of giltch and jump flagging and merit a detailed look (the literature on this specific application seems deep). Will try to get something worthy of a PR cobbled together in the coming weeks.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.