daphne-project / daphne-project/daphne

Vectorized pipelines with individual result types

Open
#397 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
81
Forks
83
PR merge metrics
No merged PRs in 30d

Description

At the moment, we require all results of a vectorized pipeline to have the same data type and value type. We should relax this requirement to allow each result to be represented in the most suitable way.

One approach to achieve this is to refactor `MTWrapper` and all related components for the superclass `Structure` (or at least `Matrix`, since we don't support `Frame`s in the vectorized engine yet) as the output type. Special behavior required for a particular data type could be implemented by the data type itself (and be accessible via the `Structure`/`Matrix` interface). This would also facilitate extensibility with new data types.

Contributor guide

Open the contributing guide

Research direction

Start by tracing the vectorized engine's use of MTWrapper and the Structure and Matrix interfaces; the issue notes that Frame is not currently supported. Determine how result data types are enforced today and define what must change so each result can use a suitable representation while preserving extensibility for new data types.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
data
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.