apache / apache/pinot

Distribute service tokens to pinot's components

Open
#17,487 2 comments 0 reactions 0 assignees View on GitHub
security troubleshooting
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
2d 55m
Merged PRs (30d)
182

Description

#### __Do we need service tokens with HDFS Kerberos?"__

__YES - Because they serve different security layers:__

__HDFS Kerberos (Already Working):__

```javascript
Controller → [Kerberos] → HDFS (File operations)
Server → [Kerberos] → HDFS (File operations)
```

__Service Tokens (Our Implementation):__

```javascript
Controller → [HTTP Token] → Server (API calls)
Controller → [HTTP Token] → Minion (API calls)
Server → [HTTP Token] → Controller (API calls)
Minion → [HTTP Token] → Controller (API calls)
```

#### __📋 Detailed Service Token Distribution:__

__Controller Configuration:__

```properties
# Keep existing Kerberos (HDFS access)
hadoop.security.authentication=kerberos

# Add service token (HTTP API access)
controller.segment.fetcher.auth.token=
```

__Server Configuration:__

```properties
# Keep existing Kerberos
hadoop.security.authentication=kerberos

# Add service tokens
pinot.server.segment.fetcher.auth.token=
pinot.server.segment.uploader.auth.token=
pinot.server.instance.auth.token=
```

__Minion Configuration:__

```properties
# Keep existing Kerberos
hadoop.security.authentication=kerberos

# Add service tokens
segment.fetcher.auth.token=
task.auth.token=
```

Contributor guide

Open the contributing guide

Research direction

Start by tracing the Controller, Server, and Minion authentication configuration, including the listed segment fetcher, uploader, instance, and task token properties, alongside the existing HDFS Kerberos settings. Define how tokens should be distributed for each HTTP API path shown and verify that all listed component-to-component calls are covered.

Written by the indexing model from the issue text.

Assessment

Tech stack
hadoop, java
Domain
authentication, backend, distributed-systems, security
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.