[FEATURE] Hybrid Compression with Zstandard Codecs

Is your feature request related to a problem?

Coming here from https://github.com/opensearch-project/OpenSearch/issues/11605#issuecomment-2046521026

We are observing good benefits from enabling Hybrid Compression during merges with default codec. That being said, I believe we would see more benefits with Zstandard since it has much better compression ratio that default codec, so the trade offs on disk throughput / iops should be reduced.

Since we would not need to introduce a new codec to implement hybrid compression with Zstandard, I propose to have an opt in settings that allows users to manage the thresholds for hybrid compression.

What solution would you like?

Hybrid Compression for Zstandard where the recently added stored fields are not compressed, which enables faster indexing, look ups for recent documents, but the stored fields are compressed upon merges.

Do you have any additional context?

Detailed Description: https://github.com/opensearch-project/OpenSearch/issues/13110

Start by reading the linked OpenSearch issues 11605 and 13110, then inspect the existing hybrid-compression implementation in this custom-codecs repository. The issue names no files or tests; done means an opt-in Zstandard hybrid-compression setting supports configurable thresholds and compresses recently added stored fields during merges.