CTL API proposal

CTL

API proposal

Intention of a CTL for the UMF is to provide path-oriented method to library control and metrics acquisition. Current implementation of the CTL internal rely on Piotr's CTL taken from PDMK.

General API

Global API should looks like:

umf_result_t umfCtlSet(const char *name, ...);
umf_result_t umfCtlGet(const char *name, ...);
umf_result_t umfCtlExec(const char *name, ...);

The name is a path-like string that traverse through the structure tree (e.g. umf.pool.default.log.enable). Rest of the arguments depends on what name should handle, and it will be covered in next chapter.

Distinguish the context - optional

Each entity should have distinct magic header to differentiate that context is either pool, provider or something else.

typedef struct {
    uint32_t magic; // Magic number to identify the type
    int field1;
    float field2;
} StructType1;

Name

Correct name should meet the following regexp:

^[a-zA-Z0-9]+((\.[a-zA-Z0-9]+)|(\.{}))*$

Context-free grammar:

S → A B
B → ε | A B |. A B | . {} B
A → 'a' | 'b' | 'c' | ... | 'z' | 'A' | 'B' | 'C' | ... | 'Z' | '0' | '1' | '2' | ... | '9'

Examples of accepted strings:

debug.log.enable
provider.purge_force
umf.pool.{}.set_arena
umf.provider.jemalloc.arenas.bin.1.size

Arg

Raw pointer to provide or retrieve data from the callback function implemented in specific module (pool/provider/etc), e.g.:

static int CTL_WRITE_HANDLER(debug_test)(void *ctx, enum ctl_query_source source,
                                        void *arg,
                                        struct ctl_index_utlist *indexes) {
    /* suppress unused-parameter errors */
    (void)source, (void)indexes;

    unsigned int arg_in = *(unsigned int *)arg;
    os_memory_provider_t *os_provider = (os_memory_provider_t *)ctx;
    os_provider->protection = arg_in;
    return 0;
}

Global namespace

User can set up default values using string , e.g.:

umfCtlSet("umf.pool.%s.min_page_size", "disjoint", &new_value); // set default value for disjoint pool
umfCtlGet("umf.pool.%s.max_page_size", "disjoint", &value); // get default value

Another way to set up global namespace is passing values by an environment variable (TBD).

Global namespace can be used for storing values for future creation of pools/providers or set up common features (e.g.: enabling logging mechanism or logging level). Storing configuration for common features is similar to operating on a given context (existing pools/providers). Storing default values for non-existent pools/providers forces to using some kind of key-value store and then retrieve matching pairs at the creation time during initialize() function.
Examples of using global namespace:

// Change default for future providers of the given type
UmfCtlSet(NULL, "provider.<provider_name>.always_zero_memory", &newvalue);

// Change value for existent provider using handle
// Notice: there is a handle provided as a context
umfCtlSet(osMemoryProviderHandle, "always_zero_memory", &newvalue);
// But user can use full path if it is preferred 
umfCtlSet(osMemoryProviderHandle, "provider.<provider_name>.always_zero_memory", &newvalue);

Options tree

TODO: provide tree structure with possible options that will be implemented

umf
├── common
│   └── TODO
├── pool
|   ├── by_handle
|   |   └── TODO
|   └── default
|       └── TODO
└── provider
    ├── by_handle
    └── default

Internal disjoint pool tree:

disjoint
├── SlabMinSize (size_t)
├── MaxPoolableSize (size_t)
├── Capacity (size_t)
├── MinBucketSize (size_t)
├── CurPoolSize (size_t)
├── PoolTrace (int)
└── Name (char *)

Invocation to set SlabMinSize should looks like:

umfCtlSet("umf.pool.default.SlabMinSize", "disjoint", &value);

Modification needed for this approach

TODO: what extra modifications are needed to make it works

Provider

1. Internal functions declaration to handle set/get/exec in include/umf/memory_provider.h

umf_result_t
umfMemoryProviderCtlGet(umf_memory_provider_handle_t hProvider, const char *name,
                     void *arg);

umf_result_t
umfMemoryProviderCtlSet(umf_memory_provider_handle_t hProvider, const char *name,
                     void *arg);

umf_result_t umfMemoryProviderCtlExec(umf_memory_provider_handle_t hProvider,
                                  const char *name, void *arg);

2. Add ctl function pointer declaration in umf_memory_provider_ext_ops_t (memory_provider_ops.h)

umf_result_t (*ctl)(void *hProvider, int operationType, const char *name, void *arg);

3. Example of function used for initialization and execute given path

static umf_result_t
os_ctl(void *hProvider, int operationType, const char *name, void *arg) {
    os_memory_provider_t *os_provider = (os_memory_provider_t *)hProvider;
    utils_init_once(&ctl_initialized, initialize_debug_ctl);
    ctl_query(ctl_debug, os_provider, CTL_QUERY_PROGRAMMATIC,
              name, operationType, arg);
    return UMF_RESULT_ERROR_NOT_SUPPORTED;
}

4. Example of implementation of READ/WRITE/EXEC handlers

static int CTL_WRITE_HANDLER(debug_test)(void *ctx, enum ctl_query_source source,
                                        void *arg,
                                        struct ctl_index_utlist *indexes) {
    /* suppress unused-parameter errors */
    (void)source, (void)indexes, (void)ctx;

    unsigned int arg_in = *(unsigned int *)arg;
    os_memory_provider_t *os_provider = (os_memory_provider_t *)ctx;
    os_provider->protection = arg_in;
    return 0;
}

Steps

1. Implementation for by_handle variant #1062
2. Variadic argument implementation #1085
3. Global namespace implementation #1058
4. Add more pools and providers to be supported by the CTL

Pull request related

CTL: Add a CTL sources to the UMF #913

Issues related

UMF: use CTL/env to allow changing default jemalloc pool options #1019

UMF: Performance Testing #786

UMF: Expose allocation statistics for memory provideres (and pools?) #282

UR: Control and instrospection API #1454