lincc-frameworks / lincc-frameworks/nested-pandas

NestedFrame Representation for Image Columns

Open
#515 0 comments 0 reactions 0 assignees View on GitHub
enhancement LSDB modalities-support
Dominant language
Python
Stars
26
Forks
8
Avg merge
2d 2h
Merged PRs (30d)
9

Description

This ticket is motivated by the MultiModal Universe/DOE Genesis work, see more information in the related tracker issue (https://github.com/astronomy-commons/lsdb/issues/1520). As NestedFrame is the in-memory view of a Catalog, it likely makes sense for nested-pandas to undertake adding support for image columns, even if just in a limited capacity. The details of the exact implementation are still to be decided, but the minimum requirements on the nested-pandas side would be:
* Basic API support; ability to load and shepherd images through to map_rows style operations. To some extent this is already achievable.
* A human-readable html representation, similar to traditional nested columns
* Defining a standard internal representation of an image, with consideration for understanding how to treat RGB images, cutouts, different input formats (byte streams, paths to image files, fits)

For a more complete requirement set:
* We would likely want to implement a custom ImageSeries which would be a parallel for NestedSeries, rather than try to expand NestedSeries, unless we opt to try to flatten images and enforce a more offset style approach similar to NestedSeries
* Come up with an approach for parquet writing with images, not necessarily serializing them, but storing them in a consistent way that is easily readable.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the related tracker issue and the existing NestedFrame and NestedSeries APIs. Determine the image representation and scope for loading, map_rows operations, HTML display, and parquet handling, including RGB images, cutouts, byte streams, paths, and FITS inputs. Done requires an agreed design and implementation plan covering these minimum requirements.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.