arviz-devs / arviz-devs/InferenceObjects.jl

Allowing alternative InferenceData/Dataset storage

Open
#16 1 comment 0 reactions 1 assignee Claimed by @sethaxen View on GitHub
enhancement
Dominant language
Julia
Stars
15
Forks
1
Avg merge
6d 18h
Merged PRs (30d)
1

Description

Currently all `Dataset`s wrap a `Dimstack`, which in turn wraps a set of `NamedTuple`s. Likewise, `InferenceData` wraps a `NamedTuple` of `Dataset`s. As noted in #15, this makes everything but the array values themselves immutable.

An alternative is to decouple the interface for `Dataset` and `InferenceData` from the storage. e.g. we could define `NamedTupleStore`, `DictStore`, `NCDatasetStore`, etc and implement the `InferenceData`/`Dataset` interfaces for each of these. This would allow users to strongly type everything with `NamedTupleStore` if they want to, or they could use a `DictStore` to have much more dynamic access. Having a `NCDatasetStore` would allow users to open a NetCDF file as an `InferenceData` and even incrementally write to such an `InferenceData`, having the NetCDF file automatically updated.

With a well-designed API for stores, this would actually look a lot like the InferenceData API proposed in https://github.com/arviz-devs/ArviZ.jl/issues/154. Namely, the store API could be implemented for a type like `MCMCChains.Chains`, and then one could call `InferenceData(chains)` to (inefficiently) view it as an `InferenceData`. When efficiency is needed, one just converts to one of the native stores.

I started work on a small prototype of this. It's a fair amount of surgery to the existing code and increases the code complexity. Though it will be more work to do it then than now, I think it's better to hold off on this until later.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.