lincc-frameworks / lincc-frameworks/rubin-dash
Ensure ZSTD compression is used for final catalogs
Open
Beginner friendly
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
In the scripts ran to correct the DP2 imports for release, the compression was reverted from ZSTD to SNAPPY. In future releases, ZSTD should be used as we have proven it performs better for catalog data.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Inspect the scripts used to correct the DP2 imports for release and find where catalog compression is selected. Confirm that the final catalogs use ZSTD rather than SNAPPY, and verify the release import flow produces catalogs with the intended compression.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data-engineering, release
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 64/100