google / google/sedpack

Allow arbitrary shard iteration

Open
#200 0 comments 0 reactions 1 assignee Claimed by @kralka View on GitHub
enhancement
Dominant language
Python
Stars
37
Forks
13
Avg merge
15h 10m
Merged PRs (30d)
11

Description

Allow low-level selection of shard iteration strategy.

https://github.com/google/sedpack/pull/198 is the first step for `as_numpy_iterator`. Then also other functions `as_numpy_*` and `as_tfdataset`.

Then support for advanced iteration strategies.

Then code cleanup of `dataset_iteration.py`.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.