simonsobs / simonsobs/socs

Additional Realism needed for Smurf File Emulator

Open
#486 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Python
Stars
17
Forks
20
Avg merge
1h 22m
Merged PRs (30d)
2

Description

Ok I've got two changes that need to be made to the smurf file emulator to better re-create what we have in the real data streams.

The big one, that's causing issues with testing, is that every file has "end frames" saved to them when they close out. With the real system, those end frames are only placed at the end of observations and we use them to determine if an observation has been completed or not. Basically, I need sotodlib.io.load_smurf._file_had_end_frames(file_name) to only return True if an observation is complete. Right now this means observations are being completed too early and the E2E system is getting a lot of hanging "stray" files that shouldn't be there.

The less big one, but if we're doing another pass on these agents we might as well, is that I would like uxm_relock to not make new tune files. Tune files (and their deletion at level 2) are a more subtle problem than for all the other types of data. It'd be nice to have fewer of them so it was harder to tell when I should be clearing them.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with sotodlib.io.load_smurf._file_had_end_frames(file_name) and the smurf file emulator implementation to trace how end frames are written and detected. Then inspect the uxm_relock agent behavior. Done means incomplete observations no longer appear complete, and uxm_relock does not create new tune files; verify the end-to-end behavior with the project’s existing tests if available.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.