oneapi-src / oneapi-src/unified-runtime

Generalize ur_usm_pool_limits_desc_t

Open
#369 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

needs-discussion specification
Dominant language
C++
Stars
57
Forks
120
Avg merge
1d 14h
Merged PRs (30d)
1

Description

Currently, ur_usm_pool_limits_desc_t exposes parameters similar to the ones exposed by SYCL_PI_LEVEL_ZERO_USM_ALLOCATOR .

There are a few problems with existing params:

  1. capacity is tricky to use: it describes a number of allocations and does not take into account allocation size. Also, UR does not expose any statistics that could be used to tweak the capacity. Moreover, it might not be applicable to all heap managers.

    I propose removing this parameter.

  2. maxPoolSize might be ambiguous. Unified Runtime can create multiple 'sub-pools' - it can decide to create pool per each device or to create a pool that spans multiple devices in certain cases. Having the same pool size limit for both of those cases will not be optimal.

    We could replace the maxPoolSize with a decay parameter that, instead of specifying limits in terms of space, defines it in terms of time - for how long the pool is allowed to keep memory pages around, similar to: https://jemalloc.net/jemalloc.3.html#arenas.dirty_decay_ms

Related discussion: https://github.com/oneapi-src/unified-runtime/pull/324

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the ur_usm_pool_limits_desc_t definition and the related discussion in PR #324; compare the existing capacity and maxPoolSize parameters with the proposed decay semantics and the linked jemalloc reference. Done means the parameter design and behavior are agreed and reflected in the Unified Runtime API.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend-api-design
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.