apache / apache/pinot

Unable to Run Ingestion Job when using ADLS Gen2 as a source and destination

Open
#11,025 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
2d 3h
Merged PRs (30d)
195

Description

My objective is to import a CSV file into Pinot. The file is currently stored in an ADLS Gen2 container. I have implemented the necessary modifications to the relevant components, as advised in the documentation provided at https://docs.pinot.apache.org/basics/data-import/pinot-file-system/import-from-adls-azure.

However, despite these adjustments, the ingestion process is encountering a failure and producing the subsequent error message:
Screenshot 2023-07-03 at 10 56 34 AM

Contributor guide

Open the contributing guide

Research direction

Start with the ADLS Gen2 import documentation linked in the issue and reproduce the CSV ingestion with ADLS Gen2 as both source and destination. Obtain the error text shown only in the screenshot, trace it to the relevant ingestion component, and verify that the CSV imports successfully.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure
Domain
cloud, data-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.