[VL] Support GPU concurrent tasks
Open
enhancement
- Dominant language
- Scala
- Stars
- 1.6k
- Forks
- 657
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 80
Description
### Description
GPU memory is less than CPU memory, so GPU concurrent thread number is less than CPU, and incase of the kernel, the GPU parallelism is a small value, make it configurable
The GPU memory pool is initialized half of the free memory for each thread, so the thread number is related to the memory initialization.
### Gluten version
None
Contributor guide
Research direction
Start by tracing where the GPU memory pool is initialized and how its per-thread allocation determines concurrent task count. Identify the existing configuration entry point, then verify that GPU concurrency can be set independently and that the configured value is respected during initialization.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- scala
- Domain
- performance
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100