scverse / scverse/spatialdata

Proposal: simplify data model for Points

Open
#503 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

element: points ⊙ element: table 📑
Dominant language
Python
Stars
394
Forks
95
Avg merge
4d 3h
Merged PRs (30d)
7

Description

While writing tests for the incremental IO of metadata I added tests also for those metadata that we originally planned in the design doc for points but that we have only rarely used.

Specifically, points have two optional values

element.attrs[PointsModel.ATTRS_KEY][PointsModel.FEATURE_KEY] = 'genes'
element.attrs[PointsModel.ATTRS_KEY][PointsModel.INSTANCE_KEY] = 'instance_id'

which specify the name of two columns with particular semantic:

  • the feature_key column is an optional column that contains the "main feature" of the points, ideally the gene_id;
  • the instance_key column is an optional column that can be used to tell which point fall into which cell.

The two columns above are very rarely used: I think that we never use instance_key and that feature_key is used in aggregate() to infer a default value for value_key when this is not specified by the user.

Since the practical rare usage I would consider simplify the points model and drop them; I think the effort to support and document a more complex data model is not paid off in practical terms in these cases. And while it's just little pieces of metadata, in general I think the leaner the model the better.

To be more precise, I have a preference for dropping both but I have no strong opinions for feature_key. Instead, I believe that we should remove instance_key for the following reasons:

  • it was added before multiple tables: now instance_key alone is not indicative of which table would contain the cells that the points refer to;
  • it was added before a table could annotate points: now, if a table annotates point, the instance_key of the table would refer to the index of the points, while the instance_key of the points would refer to the instance_key column of a different table.

Finally, an argument for dropping feature_key is that if we unify points and circles, either both should have the extra column either none, and since it's not something need for circles and it is not essential for points, I would just drop it.

CC @giovp @kevinyamauchi @melonora

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing PointsModel, aggregate(), the points design doc, and the incremental IO metadata tests mentioned in the issue. Clarify whether feature_key and instance_key should be removed, then determine the model, metadata handling, tests, and documentation that must agree with the decision.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.