opensearch-project / opensearch-project/OpenSearch
Support different compression types for snapshoting
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 13.7k
- Forks
- 3k
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 108
Description
Is your feature request related to a problem? Please describe.
For now Opensearch supports only deflate compression for snapshoting (compress setting for all Cloud plugins).
It would be great to extend compression to use different compression algorithms like:
ZSTDLZ4
or even more.
Describe the solution you'd like
To support different compression types by
extending the repository settings for compression using such JSON:
{
...
"settings": {
"compression": {
"type": "zstd", // could be `deflate`, `xz4` etc.
"compression_settings": {
// different compression settings like level and so on
}
}
}
}
In this case it is difficult to keep backward compatibility
Describe alternatives you've considered
Compression type could be set when the repository is created/updated using such JSON:
{
...
"settings": {
"compress": true,
"compression_type": "zstd", // `deflate`, `lz4`
}
}
and add compression settings in opensearch.yml something like:
compression.<propery>: <some_property_value>
In this case it is possible to keep backward compatibility between versions. e.g. deflate is a default type for compression
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the snapshot repository settings and the existing deflate handling in the OpenSearch codebase, then review how Cloud plugins and opensearch.yml currently configure compression. Compare the proposed repository-level and configuration-level approaches, including backward compatibility. Done means supported compression types and settings are defined with a compatible default and covered by the relevant tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100