apache / apache/uniffle

[Problem] The shuffle server memory not release

Open
#233 11 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
454
Forks
172
Avg merge
5d 17h
Merged PRs (30d)
5

Description

uniffle version: 0.6.0, I deploy on k8s,but the shuffle server memory not release ,even my spark application is stop it does not release too
my server config is below:
> rss.rpc.server.port 19999
rss.jetty.http.port 19998
rss.rpc.executor.size 2000
rss.storage.type MEMORY_LOCALFILE_HDFS
rss.coordinator.quorum 10.100.41.162:19999
rss.server.disk.capacity 50g
rss.storage.basePath /home/data
rss.server.flush.thread.alive 1
rss.server.flush.threadPool.size 10
rss.server.buffer.capacity 4g
rss.server.read.buffer.capacity 2g
rss.server.heartbeat.timeout 60000
rss.server.heartbeat.interval 10000
rss.rpc.message.max.size 1073741824
rss.server.preAllocation.expired 120000
rss.server.commit.timeout 600000
rss.server.app.expired.withoutHeartbeat 120000
rss.server.flush.cold.storage.threshold.size 128m

![image](https://user-images.githubusercontent.com/22764286/191216608-f04adaca-81fd-4a73-919d-0ea36f0657bc.png)

Contributor guide

Open the contributing guide

Research direction

The report names no source file, test, or entry point; begin by reviewing the supplied Uniffle server configuration and the memory behavior after the Spark application stops. Done means identifying why the shuffle server retains memory and confirming that it is released after the application ends.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, kubernetes, spark
Domain
backend, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.