/ingestFromFile rest call fails to upload segment if user name password is enabled Pinot 0.9.2
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 195
Description
Looks like auth token needs to be passed through when calling the controller API
**/ingestFromFile**
which calls
**/segments** api
```
Caused by: org.apache.pinot.common.exception.HttpErrorStatusException: Got error status code: 403 (Forbidden) with reason: "Permission is denied for access type 'CREATE' to the endpoint 'http://somehost:9000/v2/segments' for table 'dimSomeTable_OFFLINE'" while sending request: http://somehost:9000/v2/segments?tableName=dimSomeTable to controller: somehost, version: Unknown
at org.apache.pinot.common.utils.FileUploadDownloadClient.sendRequest(FileUploadDownloadClient.java:510) ~[pinot-all-0.9.2-jar-with-dependencies.jar:0.9.2-3e5e5aca07ad796d453ad
d80a2b2b1bf1d654cf1]
at org.apache.pinot.common.utils.FileUploadDownloadClient.uploadSegment(FileUploadDownloadClient.java:791) ~[pinot-all-0.9.2-jar-with-dependencies.jar:0.9.2-3e5e5aca07ad796d453
add80a2b2b1bf1d654cf1]
at org.apache.pinot.segment.local.utils.SegmentPushUtils.lambda$pushSegments$0(SegmentPushUtils.java:120) ~[pinot-all-0.9.2-jar-with-dependencies.jar:0.9.2-3e5e5aca07ad796d453a
dd80a2b2b1bf1d654cf1]
at org.apache.pinot.spi.utils.retry.BaseRetryPolicy.attempt(BaseRetryPolicy.java:50) ~[pinot-all-0.9.2-jar-with-dependencies.jar:0.9.2-3e5e5aca07ad796d453add80a2b2b1bf1d654cf1]
at org.apache.pinot.segment.local.utils.SegmentPushUtils.pushSegments(SegmentPushUtils.java:117) ~[pinot-all-0.9.2-jar-with-dependencies.jar:0.9.2-3e5e5aca07ad796d453add80a2b2b
1bf1d654cf1]
at org.apache.pinot.segment.local.utils.IngestionUtils.uploadSegment(IngestionUtils.java:204) ~[pinot-all-0.9.2-jar-with-dependencies.jar:0.9.2-3e5e5aca07ad796d453add80a2b2b1bf
1d654cf1]
... 28 more
2022/07/06 23:31:00.893 ERROR [PinotIngestionRestletResource] [jersey-server-managed-async-executor-5] Caught exception when ingesting file into table: dimSomeTable_OFFLINE. Caught
exception while uploading segments. Push mode: TAR, segment tars: [[file:/local/data/logs/s3:/blah/upload_
dir/working_dir_dimSomeTable_OFFLINE_1657164660862/segment_tar_dir/dimSomeTable_1657164660863.tar.gz]]
java.lang.RuntimeException: Caught exception while uploading segments. Push mode: TAR, segment tars: [[file://local/data/logs/s3:/blah/upload_
dir/working_dir_dimSomeTable_OFFLINE_1657164660862/segment_tar_dir/dimSomeTable_1657164660863.tar.gz]]
at org.apache.pinot.segment.local.utils.IngestionUtils.uploadSegment(IngestionUtils.java:207) ~[pinot-all-0.9.2-jar-with-dependencies.jar:0.9.2-3e5e5aca07ad796d453add80a2b2b1bf
1d654cf1]
```
Contributor guide
Research direction
Start at PinotIngestionRestletResource for /ingestFromFile, then trace the request through IngestionUtils, SegmentPushUtils, and FileUploadDownloadClient to the /segments controller call. Verify how authentication is carried into that upload request; done means an authenticated ingestion no longer receives the reported 403 response.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- api, authentication, backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100