The local directory of the container should be used as the temporary directory of fluss client when the Flink job is deployed to YARN
- Dominant language
- Java
- Stars
- 2.1k
- Forks
- 625
- Avg merge
- 3d 14h
- Merged PRs (30d)
- 97
Description
### Search before asking
- [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar.
### Fluss version
0.8.0 (latest release)
### Please describe the bug 🐞
I deploy a flink streaming job to YARN. This job reads some fluss tables. It's taskmanager (YARN container) use /tmp/fluss as the temporary directory. I think that the local directory of the container should typically be used as the temporary directory of fluss read client.
### Solution
Use one directory in the option 'io.tmp.dirs' of flink configuration to set the option ‘client.scanner.io.tmpdir’ of fluss configuration after the taskmanager startup
### Are you willing to submit a PR?
- [x] I'm willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start at Flink TaskManager startup and trace how the Flink configuration option `io.tmp.dirs` is made available to the Fluss client. Verify that `client.scanner.io.tmpdir` uses a directory from that setting instead of `/tmp/fluss` in YARN containers, and confirm the behavior with the relevant client and deployment tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- distributed-systems
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100