JuliaArrays / JuliaArrays/AxisArrays.jl
Roadmap
Nobody has claimed this yet.
- Dominant language
- Julia
- Stars
- 207
- Forks
- 41
- PR merge metrics
- No merged PRs in 30d
Description
We're getting to the point where the core API is stabilizing. We can now start making things fancier and building out interfaces to base methods and other packages (like #5). Here's a current brain-dump of some of my thoughts. Additions, critiques and comments are very welcome.
Remaining core infrastructure
- Vector and multi-dimensional
setindex!(mirroringgetindex's capabilities). (#11) - Type-stable constructors. We should allow passing tuples of
Axistypes to specify the dimension names. It would be an interesting experiment to store the dimension names asAxistypes instead of symbols. I'm not sure if that'd make things simpler or more convoluted. It may be a mixed bag. - Display. We should display axis names and values in a sensible way. In some regards, I think they're more important to see than a small window into the data, particularly with more than 2 dimensions. This isn't perfect, but 4b62efe8c1333e9135bbdb8d43454f241eec11fe is a pretty big improvement.
- Online documentation and a README revamp. I think that Lexicon.jl/MkDocs is currently the easiest and best solution to make the inline documentation accessible online. Basic implementation from #29.
Possible additions to the core infrastructure
-
Add a third flavor of axis trait for Dimensional axes with elements of a discrete step-like type. The key defining characteristic of this element type is that their StepRanges must enumerate all values between the endpoints, allowing us to provide sensible indexing directly with a StepRange. This also means that there's no issues along the lines of floating point instability, so we can also allow indexing directly with single-elements of this type (and don't need to force the use of Intervals). The main use-case I see here is forDate. Are there other types that satisfy these criteria? What is a sensible name for this trait? - Allow (or maybe even encourage) the use of an ordered hashmap for categorical axes to enable O(1) index lookup. Some experimentation is required here: I'm sure the linear search will outperform hashing for small N (particularly with symbols), but what's the cutoff? 10? 100? What about strings? Chances are that folks won't be using categorical vectors to enumerate more than 100 elements.
- Allow an unsafe constructor that doesn't check axis invariants. Or maybe check upon wrapping with an
Axistype (#15). Ensuring that large, non-Range Dimensional axes are monotonically increasing can take a long time (we may be able to speed this up some with a special Ordering type, but it's still O(n)). Similarly, ensuring that elements in categorical vectors are unique requires hashing all elements (which could be used for the above hashmap). - Custom iterators. An
eachsliceiterator would be nice and useful in and of itself, and if we allow the same sort of syntax and semantics asmapslicesit can serve as the building block for augmenting that Base function. - Windowed repetitions, with a more generic implementation of
Signals.window. I was thinking of allowing windowing as an indexing operation (https://github.com/mbauman/Signals.jl/issues/10), but constructing vectors of interval types with deferred promotion (to allow, e.g., windows specified in time about integer indices) has been very challenging.
Extensions to Base
- We should specialize all Base functions that allow selecting specific dimensions, like
sum,mean,maximum,mapslices, etc. We can also return AxisArrays with the properly reduced axis set, dropping a dimension and eliminating the type-unstable squeezes. - Permutations and transposes could keep track of and preserve axis information
- NamedArrays.jl and TimeSeries.jl go even farther to specialize matrix arithmetic to try to preserve names through operations. I'm not convinced this is worth the effort, but it is appealing.
Extensions to other packages
- It'd be nice to allow re-interpolating along any axis with Interpolations.jl. This will require some work on Interpolations.jl, too.
- DSP.jl could provide filtering (https://github.com/mbauman/Signals.jl/issues/1) with sensible units for filter design and spectrograms that return a properly annotated AxisArray with an extra frequency (or inverse-whatever) axis.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
This issue is a broad roadmap, not a single implementation task. Start by selecting one unchecked item and reading the referenced Base functions, Signals.jl issues, Interpolations.jl, or DSP.jl context; scope is complete only when that item has a defined implementation and tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- data
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100