SegmentCreationAndMetadataPush may have the issue with the path
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 189
Description
There was a bug report from OSS slack where`SegmentCreationAndMetadataPush` from the latest branch may have the issue with the path.
Setup
1. Source = local
2. Destination = ADLS
3. PushType = MetadataPush
4. JobUsed = `SegmentCreationAndMetadataPush`
On the client side, they get the following error:
```
2023/03/03 15:21:07.100 ERROR [DataLakePathClient] [main] Status code 404, (BlobNotFound)
```
This doesn't happen with 0.10.0 version (`0.10.0-adls-fix` branch)
Contributor guide
Research direction
Start by reproducing the setup with local source, ADLS destination, MetadataPush, and SegmentCreationAndMetadataPush, then trace the path handling around the BlobNotFound error. Compare the behavior with the 0.10.0-adls-fix branch; done means the metadata push uses a valid ADLS path without the reported 404.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, java
- Domain
- cloud
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100