Zarr3 chunking issues
- Dominant language
- C++
- Stars
- 1.5k
- Forks
- 145
- PR merge metrics
- No merged PRs in 30d
Description
I am running into a weird error when adding chunking mixed with compression to my zarr3 file with tensorstore. This error took me a while to discover the root cause of since the error message is pretty ambiguous in my eyes. It seems that when my chunk size gets too large with compression on, I receive this error. However, based on my math (550 * 200 * 10 * 8) this should only be around a 9 mb chunk which is relatively small from my research online and within the sizes for zstd compression, so I don't understand why it is causing an error. Any insights or discussion would be appreciated.
```c++
tensorstore::Context defaultContext = tensorstore::Context::Default();
// Define the shape and data type of the tensorstore
tensorstore::Result> file = tensorstore::Open({{"driver", "zarr3"},
{"kvstore", {{"driver", "file"}, {"path", filepath}}},
{"metadata",
{
{"data_type", "float64"},
{"fill_value", 0.0},
{"shape", {1500, 700, 200, 10}},
{"codecs",
{{{"name", "zstd"}, {"configuration", {{"level", 1}}}}}},
{"chunk_grid",
{{"name", "regular"},
{"configuration",
{{"chunk_shape", {1, 550, 200, 10}}}}}},
}}},
defaultContext, tensorstore::OpenMode::delete_existing | tensorstore::OpenMode::create, tensorstore::ReadWriteMode::read_write)
.result();
```
Error:
`terminate called after throwing an instance of 'std::runtime_error'
what(): Failed to create TensorStore: Error opening "zarr3" driver: Mismatch in "codecs": Cannot merge zarr codec constraints [] and [{"configuration":{"level":1},"name":"zstd"}]: Mismatch in number of bytes -> bytes codecs (0 vs 1)
`
Contributor guide
Assessment
This issue has not been assessed yet.