apache / apache/pinot

SegmentCreationAndMetadataPush may have the issue with the path

Open
#10,377 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
1d 21h
Merged PRs (30d)
189

Description

There was a bug report from OSS slack where`SegmentCreationAndMetadataPush` from the latest branch may have the issue with the path.

Setup
1. Source = local
2. Destination = ADLS
3. PushType = MetadataPush
4. JobUsed = `SegmentCreationAndMetadataPush`

On the client side, they get the following error:

```
2023/03/03 15:21:07.100 ERROR [DataLakePathClient] [main] Status code 404, (BlobNotFound)
```

This doesn't happen with 0.10.0 version (`0.10.0-adls-fix` branch)

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the setup with local source, ADLS destination, MetadataPush, and SegmentCreationAndMetadataPush, then trace the path handling around the BlobNotFound error. Compare the behavior with the 0.10.0-adls-fix branch; done means the metadata push uses a valid ADLS path without the reported 404.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, java
Domain
cloud
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.