[CORE] bug report: collision of resource profile id
- Dominant language
- Scala
- Stars
- 44k
- Forks
- 29.4k
- PR merge metrics
- No merged PRs in 30d
Description
Spark's default resource profile id is 0. In stage level resource profile, the first returned resource id is also 0 which would lead to a collision.
```
val UNKNOWN_RESOURCE_PROFILE_ID = -1
val DEFAULT_RESOURCE_PROFILE_ID = 0
private lazy val nextProfileId = new AtomicInteger(0)
```
https://github.com/apache/spark/blob/f8836939c6a89bd788ad648ec30df757cde204ac/core/src/main/scala/org/apache/spark/resource/ResourceProfile.scala#L365
https://github.com/apache/spark/blob/f8836939c6a89bd788ad648ec30df757cde204ac/core/src/main/scala/org/apache/spark/resource/ResourceProfile.scala#L363
Contributor guide
Research direction
Start in core/src/main/scala/org/apache/spark/resource/ResourceProfile.scala at the DEFAULT_RESOURCE_PROFILE_ID and nextProfileId definitions linked in the issue. Trace how stage-level resource profile IDs are allocated and verify that the first generated ID cannot collide with the default ID; done means the two ID spaces remain distinct.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- scala
- Domain
- distributed-systems
- Issue type
- Bug
- Difficulty
- 1/5
- Estimated time
- Under an hour
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 82/100