Additional Realism needed for Smurf File Emulator
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 17
- Forks
- 20
- Avg merge
- 1h 22m
- Merged PRs (30d)
- 2
Description
Ok I've got two changes that need to be made to the smurf file emulator to better re-create what we have in the real data streams.
The big one, that's causing issues with testing, is that every file has "end frames" saved to them when they close out. With the real system, those end frames are only placed at the end of observations and we use them to determine if an observation has been completed or not. Basically, I need sotodlib.io.load_smurf._file_had_end_frames(file_name) to only return True if an observation is complete. Right now this means observations are being completed too early and the E2E system is getting a lot of hanging "stray" files that shouldn't be there.
The less big one, but if we're doing another pass on these agents we might as well, is that I would like uxm_relock to not make new tune files. Tune files (and their deletion at level 2) are a more subtle problem than for all the other types of data. It'd be nice to have fewer of them so it was harder to tell when I should be clearing them.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with sotodlib.io.load_smurf._file_had_end_frames(file_name) and the smurf file emulator implementation to trace how end frames are written and detected. Then inspect the uxm_relock agent behavior. Done means incomplete observations no longer appear complete, and uxm_relock does not create new tune files; verify the end-to-end behavior with the project’s existing tests if available.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100