kubeflow / kubeflow/trainer

Support for Tensor Caching in Kubeflow Data Cache

Open
#3,173 13 comments 0 reactions 1 assignee Claimed by @akshaychitneni View on GitHub
area/data-cache kind/feature
Dominant language
Go
Stars
2.2k
Forks
1.1k
Avg merge
3d 22h
Merged PRs (30d)
39

Description

### What you would like to be added?

This issue tracks caching of tokenized datasets to accelerate data loading for fine-tuning use cases. By caching tokenized data, we can avoid re-tokenizing the dataset on every GPU node and significantly improve training speed, especially for hyperparameter optimization once support for a common initializer is available.

The goal is to offload the tokenization step to data-cache CPU nodes, freeing GPU nodes to focus exclusively on training.

### Why is this needed?

Tensor caching will boost GPU utilization by enabling reuse of them across training nodes.

### Love this feature?

Give it a 👍 We prioritize the features with most 👍

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.