stan-dev / stan-dev/stan

FEATURE REQUEST: Can we have native support for ragged arrays?

Open
#3,326 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
2.8k
Forks
388
Avg merge
2d 17h
Merged PRs (30d)
15

Description

I think it would be great if there is a native support for ragged arrays in the datatypes. The need for this appears in practice, when we have groups of measurements, but each group has different size. Such problems are often suitable for hierarchical modelling for which STAN is often a good tool of choice.

The user guide suggests two approaches how to address this in the current state.

  1. the inner arrays having a known maximum size, using (possibly sparse) binary indicators to distinguish whether data is missing or available.
  2. using a flattened array with a list of indices where it should be separated.

It's quite obvious that 1) is a possibly huge waste of memory in many cases.
Technically, 2) works but it's still not great. The main issue I have with it is that it's at the cost of loss of semantics: It makes the code harder to write, read and maintain. Performance-wise, 2) still (according to the guide) makes a copy of parts of the array within model (using a function like block or segment), which is a bit of a waste, it's not terrible, but certainly not optimal if we could instead reference the data directly without copying.

Description:

My idea is to allow type declarations for array of arrays of unknown length (especially in the data section) such as:

int m, n; 
array[m] array[] int array_of_varying_length_int_arrays;
array[m] vector[] array_of_varying_length_vectors;
array[m] matrix[, n] array_of_varying_height_matrices;

Then I expect to be able to get a vector using vector[] v = array_of_varying_length_vectors[k] and obtaining it's length using size(v) or a similar function.

Not sure how difficult is this to implement, I'm just a measly user, seems probably like a bigger change.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the Stan User Guide's sparse ragged data section and compare its maximum-size and flattened-array approaches, including the mentioned block or segment functions. Define done as supporting the proposed array-of-arrays declarations, indexing into an inner array, and obtaining its length with size or a similar function.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
compilers
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.