51zero / 51zero/eel-sdk

Enhanced error handling for HiveWriter

未关闭
#358 3 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
enhancement help wanted
主要语言
Scala
星标
147
派生
32
PR 合并指标
30 天内没有已合并 PR

描述

An enhancement request if possible...

When an exception is thrown from the underlying format writer (Parquet, Orc) ... it would be nice if we could trap the exception higher up the stack in the HiveSink and report the offending column and rethrow with the column in the message.

This would greatly help when trying to find data issues coming from the source (JdbcSource).

贡献指南

这个仓库没有索引到贡献指南

调研方向

Look at the HiveSink and HiveWriter classes to understand the exception flow from the Parquet/Orc format writers. Identify where exceptions are caught and how column information can be propagated. The goal is to wrap the exception with the offending column name in the error message before rethrowing.

由索引模型根据 Issue 内容生成。

评估

技术栈
hadoop, scala
领域
backend, data-engineering
Issue 类型
功能
难度
4/5
预计耗时
3-5 天
活跃度
停滞
描述清晰度
基本清楚
新手友好度
35/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。