apache / apache/pinot

/ingestFromFile rest call fails to upload segment if user name password is enabled Pinot 0.9.2

Open
#9,027 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
2d 3h
Merged PRs (30d)
195

Description

Looks like auth token needs to be passed through when calling the controller API

**/ingestFromFile**

which calls

**/segments** api

```
Caused by: org.apache.pinot.common.exception.HttpErrorStatusException: Got error status code: 403 (Forbidden) with reason: "Permission is denied for access type 'CREATE' to the endpoint 'http://somehost:9000/v2/segments' for table 'dimSomeTable_OFFLINE'" while sending request: http://somehost:9000/v2/segments?tableName=dimSomeTable to controller: somehost, version: Unknown
at org.apache.pinot.common.utils.FileUploadDownloadClient.sendRequest(FileUploadDownloadClient.java:510) ~[pinot-all-0.9.2-jar-with-dependencies.jar:0.9.2-3e5e5aca07ad796d453ad
d80a2b2b1bf1d654cf1]
at org.apache.pinot.common.utils.FileUploadDownloadClient.uploadSegment(FileUploadDownloadClient.java:791) ~[pinot-all-0.9.2-jar-with-dependencies.jar:0.9.2-3e5e5aca07ad796d453
add80a2b2b1bf1d654cf1]
at org.apache.pinot.segment.local.utils.SegmentPushUtils.lambda$pushSegments$0(SegmentPushUtils.java:120) ~[pinot-all-0.9.2-jar-with-dependencies.jar:0.9.2-3e5e5aca07ad796d453a
dd80a2b2b1bf1d654cf1]
at org.apache.pinot.spi.utils.retry.BaseRetryPolicy.attempt(BaseRetryPolicy.java:50) ~[pinot-all-0.9.2-jar-with-dependencies.jar:0.9.2-3e5e5aca07ad796d453add80a2b2b1bf1d654cf1]
at org.apache.pinot.segment.local.utils.SegmentPushUtils.pushSegments(SegmentPushUtils.java:117) ~[pinot-all-0.9.2-jar-with-dependencies.jar:0.9.2-3e5e5aca07ad796d453add80a2b2b
1bf1d654cf1]
at org.apache.pinot.segment.local.utils.IngestionUtils.uploadSegment(IngestionUtils.java:204) ~[pinot-all-0.9.2-jar-with-dependencies.jar:0.9.2-3e5e5aca07ad796d453add80a2b2b1bf
1d654cf1]
... 28 more
2022/07/06 23:31:00.893 ERROR [PinotIngestionRestletResource] [jersey-server-managed-async-executor-5] Caught exception when ingesting file into table: dimSomeTable_OFFLINE. Caught
exception while uploading segments. Push mode: TAR, segment tars: [[file:/local/data/logs/s3:/blah/upload_
dir/working_dir_dimSomeTable_OFFLINE_1657164660862/segment_tar_dir/dimSomeTable_1657164660863.tar.gz]]
java.lang.RuntimeException: Caught exception while uploading segments. Push mode: TAR, segment tars: [[file://local/data/logs/s3:/blah/upload_
dir/working_dir_dimSomeTable_OFFLINE_1657164660862/segment_tar_dir/dimSomeTable_1657164660863.tar.gz]]
at org.apache.pinot.segment.local.utils.IngestionUtils.uploadSegment(IngestionUtils.java:207) ~[pinot-all-0.9.2-jar-with-dependencies.jar:0.9.2-3e5e5aca07ad796d453add80a2b2b1bf
1d654cf1]
```

Contributor guide

Open the contributing guide

Research direction

Start at PinotIngestionRestletResource for /ingestFromFile, then trace the request through IngestionUtils, SegmentPushUtils, and FileUploadDownloadClient to the /segments controller call. Verify how authentication is carried into that upload request; done means an authenticated ingestion no longer receives the reported 403 response.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
api, authentication, backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.