apache / apache/spark

[CORE] bug report: collision of resource profile id

Open Beginner friendly
#57,498 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Scala
Stars
44k
Forks
29.4k
PR merge metrics
No merged PRs in 30d

Description

Spark's default resource profile id is 0. In stage level resource profile, the first returned resource id is also 0 which would lead to a collision.

```
val UNKNOWN_RESOURCE_PROFILE_ID = -1
val DEFAULT_RESOURCE_PROFILE_ID = 0

private lazy val nextProfileId = new AtomicInteger(0)
```
https://github.com/apache/spark/blob/f8836939c6a89bd788ad648ec30df757cde204ac/core/src/main/scala/org/apache/spark/resource/ResourceProfile.scala#L365

https://github.com/apache/spark/blob/f8836939c6a89bd788ad648ec30df757cde204ac/core/src/main/scala/org/apache/spark/resource/ResourceProfile.scala#L363

Contributor guide

Open the contributing guide

Research direction

Start in core/src/main/scala/org/apache/spark/resource/ResourceProfile.scala at the DEFAULT_RESOURCE_PROFILE_ID and nextProfileId definitions linked in the issue. Trace how stage-level resource profile IDs are allocated and verify that the first generated ID cannot collide with the default ID; done means the two ID spaces remain distinct.

Written by the indexing model from the issue text.

Assessment

Tech stack
scala
Domain
distributed-systems
Issue type
Bug
Difficulty
1/5
Estimated time
Under an hour
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
82/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.