apache / apache/arrow

Substrait SetRel

Open
#13,967 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 18h
Merged PRs (30d)
91

Description

I would like to use the SetRel operator to union two datasets. I got the error: `NotImplemented: conversion to arrow::compute::Declaration from Substrait relation set{...}`. According to the `relation_internal.cc`, it looks like we have not implemented this operator. Is there any way to address this problem or any plan to implement the operator in the future?

The Substrait plan is as the following:
[myplan.txt](https://github.com/apache/arrow/files/9418427/myplan.txt)

Contributor guide

Open the contributing guide

Research direction

Start by reading relation_internal.cc and the attached myplan.txt to understand how the Substrait SetRel plan reaches conversion to arrow::compute::Declaration. Trace the existing relation conversion paths and determine what is needed for the reported union of two datasets. Done means the supplied plan converts successfully without the current NotImplemented error.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
data
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.