apache / apache/seatunnel

[Feature] Decouple SeaTunnel Client and Server SEATUNNEL_HOME

Open
#10,167 4 comments 0 reactions 1 assignee Claimed by @LiJie20190102 View on GitHub
help wanted
Dominant language
Java
Stars
9.7k
Forks
2.4k
Avg merge
3d 9h
Merged PRs (30d)
204

Description

## 1. Current Problem

Today SeaTunnel assumes that client and server nodes use the **same** `SEATUNNEL_HOME` path. This creates deployment inflexibility:

- **Client** : `/home/user/seatunnel-client`
- **Server** : `/opt/seatunnel-server`

When paths differ, code breaks because:
- Client may send its absolute paths to server
- Server expects files at locations that don't exist
- Ops teams forced to use identical paths everywhere

---

## 2. Goal

**Allow client and server to use different `SEATUNNEL_HOME` values**, while each maintains the standard directory structure internally:

```
Client: /home/user/seatunnel-client/
├── bin/
├── connectors/
├── config/
└── lib/

Server: /opt/seatunnel-server/
├── bin/
├── connectors/
├── config/
└── lib/
```

**Key principle**: Each process resolves paths based on **its own** `SEATUNNEL_HOME`, not paths from other nodes.

---

## 3. Requirements

### 3.1 Core Requirements

1. Client and server can use **different root paths**
2. Internal structure (`connectors/`, `config/`, `lib/`) remains **fixed relative to `SEATUNNEL_HOME`**
3. Client only sends **logical job configs** (connector names, not absolute paths) to server
4. Each side loads resources from **its own directories**
5. **Full backward compatibility**: existing deployments (same path everywhere) work unchanged

---

## 4. Implementation Approach

### 4.1 Core Principle

**Each process (client/server) resolves paths independently based on its own `SEATUNNEL_HOME` environment variable.**

### 4.2 Key Changes

1. **Unified Path Resolution**
- Create a `PathResolver` utility that reads `SEATUNNEL_HOME` and resolves standard subdirectories
- Replace all hardcoded path concatenations with calls to `PathResolver`

2. **Client-Server Communication**
- Client sends **logical references** (e.g., connector name `"kafka-source"`) in job configs
- Server resolves connector from its own `$SEATUNNEL_HOME/connectors/` directory
- **Never** pass absolute filesystem paths from client to server

3. **Backward Compatibility**
- If `SEATUNNEL_HOME` is same everywhere (current setup), behavior unchanged
- No configuration file changes needed

### 4.3 What Needs Fixing

Identify and fix code that:
- Assumes client and server have same `SEATUNNEL_HOME`
- Passes absolute paths from client to server (e.g., in job submission)
- Hardcodes path concatenation like `System.getenv("SEATUNNEL_HOME") + "/connectors"`

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.