simonsobs / simonsobs/sotodlib
Wavelet Jumps
@skhrg is already working on this.
Since Apr 10, 2024.
- Dominant language
- Python
- Stars
- 19
- Forks
- 23
- Avg merge
- 1d 5h
- Merged PRs (30d)
- 14
Description
Mostly just a looming todo item for me so that I can feel ok about putting this on the back burner and work on more pressing things.
Finding jumps with a matched filter seems to mostly work but there are some issues namely:
- A fourier space convolution doesn't really like the sharp discontinuity
- Real space convolution are slow...
But wavelets make a lot of sense here since they are good at dealing with these sorts of localized features.
A naive attempt of looking at the CWT with a gaus1 wavelet seems to work well, but CWTs are probably more expensive than we need. This paper uses a DWT dydatic scales and seems to be the basis of a lot of modern approaches (though I cannot find a python implementation of the wavelet they use). This paper is a good followup with more information about dealing with noise.
Regardless of the exact technique used I think my takeaway is that using methods like this to then get an estimate of the Lipshitz regularity would be a very powerful technique in our flagging toolkit (since that also get us glitches at the same time but in a way where we know what is what). This paper is a good overview of estimating alpha.
I can imagine two modes here that we could use:
- Estimate alpha at each point in the TOD and then use that to flag jumps and gltiches.
- Do a cheap but perhaps false positive prone selection (ie: a thresh on the gradient, this can be done in the wavelett basis as well) and then estimate alpha in each of the flagged zones to kill false positives.
Overall I think that these sorts of wavelet techniques have the potential to be the final boss of giltch and jump flagging and merit a detailed look (the literature on this specific application seems deep). Will try to get something worthy of a PR cobbled together in the coming weeks.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.