AbsaOSS / AbsaOSS/spline-spark-agent

Spline support for expand operation

Abierto
#814 0 comentarios 1 reacción 0 asignados Ver en GitHub
feature
Lenguaje dominante
Scala
Estrellas
210
Forks
102
Merge medio
1 d 1 h
PR fusionados (30 d)
1

Descripción

## Background

The Spline agent does not support column-level lineage tracking in expand operations. So, if my sql contains "grouping", or "with cube", the column lineage will be lost after expand.
## Feature

Add OperationNodeBuilder to deal with spark expand operation.

## Example [Optional]

![image](https://user-images.githubusercontent.com/8408505/233048088-fc85180a-18f1-43a7-a9ce-fe093d248cab.png)

## Proposed Solution [Optional]

Solution Ideas

1. I add a ExpandNodeBuilder class in package za.co.absa.spline.harvester.builder.plan, which has almost the same code as UnionNodeBuilder
2. I modified the class OperationNodeBuilderFactory, to deal with expand operators.

Guía de contribución

No hay ninguna guía de contribución indexada para este repositorio

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.