google/grain

View on GitHub

Library for reading and processing ML training data.

Stars: 781
Forks: 86
Open beginner issues: 0
Indexed issues: 41
Avg merge: 2d 6h
Merged PRs (30d): 6

Dominant language: Python
License: Apache-2.0
Last GitHub push: Sep 16, 2026
Latest indexed: Sep 16, 2026
Contributing guide: Contributing guide
Code of conduct: Code of conduct
Beginner labels: No beginner labels indexed

Beginner friendly Everything

Open All

41 issues indexed so far

ThreadPrefetch `close()` cannot cancel a producer blocked in `parent.__next__()` Open

type:bug

Difficulty 4/5 3-5 days Newbie friendliness 68/100

google/grain#1372 · Opened Aug 2, 2026
How are config options meant to be set in Grain? Open

type:support

Difficulty 3/5 1-2 days Newbie friendliness 45/100

google/grain#1360 · Opened Jul 15, 2026
Sliding window operation in `IterDataset` / build grain locally Open

type:support

Difficulty 4/5 3-5 days Newbie friendliness 48/100

google/grain#1264 · 1 comment · Opened Mar 26, 2026
No support for python 3.13t and 3.14t Open

type:feature

Difficulty 4/5 3-5 days Newbie friendliness 35/100

google/grain#1245 · 1 comment · 3 reactions · Opened Mar 17, 2026
Issue on page /grain.experimental.html Open

documentation

Difficulty 4/5 3-5 days Newbie friendliness 35/100

google/grain#1238 · 4 comments · 1 reaction · Opened Mar 7, 2026
Add Windows Build to `conda-forge` Open

type:feature

Difficulty 4/5 3-5 days Newbie friendliness 38/100

google/grain#1232 · Opened Feb 27, 2026
Grain workers use GPUs, and there is no way to avoid it Open

type:performance

Difficulty 4/5 3-5 days Newbie friendliness 35/100

google/grain#1199 · 5 comments · Opened Jan 17, 2026
`ThreadPrefetchIterDataset` memory leaks / zombie threads that hang exiting? Open

type:bug

Difficulty 4/5 3-5 days Newbie friendliness 45/100

google/grain#1196 · 3 comments · Opened Jan 15, 2026
Performance gap with `tf.data.Dataset` on MNIST Open

type:performance

Difficulty 4/5 3-5 days Newbie friendliness 45/100

google/grain#1164 · 5 comments · Opened Dec 15, 2025
random and augmentation in datasets Open

type:support

Difficulty 4/5 3-5 days Newbie friendliness 25/100

google/grain#1102 · 1 comment · Opened Oct 29, 2025
grain dataset slow with huggingface data source Open

type:performance

Difficulty 4/5 3-5 days Newbie friendliness 30/100

google/grain#1084 · Opened Oct 16, 2025
Are individual contributions being reviewed? Closed

type:support

Difficulty 5/5 Over a week Newbie friendliness 15/100

google/grain#1063 · 1 comment · 1 reaction · Opened Sep 29, 2025
Cannot use grain with pyarrow on Mac M1 Pro Closed

type:bug

Difficulty 3/5 1-2 days Newbie friendliness 35/100

google/grain#1045 · 2 comments · Opened Sep 18, 2025
Leaked shared_memory objects to clean up at shutdown Open

type:bug

Difficulty 4/5 3-5 days Newbie friendliness 45/100

google/grain#1031 · 1 comment · 2 reactions · Opened Sep 3, 2025
Mutex lock conflict between datasets and grain libraries with array_record>=0.8 Closed

type:bug

Difficulty 4/5 3-5 days Newbie friendliness 35/100

google/grain#1023 · 1 comment · Opened Sep 2, 2025
`ConcatThenSplitIterDataset` producing incorrect results Closed

type:bug

Difficulty 3/5 1-2 days Newbie friendliness 42/100

google/grain#981 · 2 comments · Opened Aug 10, 2025
`MapDataset` does not respect `jax.default_device(jax.devices("cpu")[0])`. Open

type:bug

Difficulty 3/5 1-2 days Newbie friendliness 35/100

google/grain#975 · 3 comments · 1 reaction · Opened Aug 6, 2025
Add dataloader checkpointing example to docs Open

documentation type:feature

Difficulty 2/5 1-3 hours Newbie friendliness 45/100

google/grain#938 · 2 comments · Opened Jul 10, 2025
DataLoader ignores context manager for Data Sources Open

type:support

Difficulty 3/5 1-2 days Newbie friendliness 62/100

google/grain#936 · Opened Jul 5, 2025
Document backwards compatibility guarantees Open

type:feature

Difficulty 4/5 3-5 days Newbie friendliness 42/100

google/grain#898 · 1 comment · Opened Jun 5, 2025
Feature: Define transforms via Protocol instead of abstract class Open

type:feature

Difficulty 4/5 3-5 days Newbie friendliness 45/100

google/grain#884 · 2 comments · Opened May 28, 2025
Discussion: Grain packaging nightly builds Open

type:build/install

Difficulty 5/5 Over a week Newbie friendliness 25/100

google/grain#878 · 2 comments · Opened May 21, 2025
SharedMemoryArray is too slow Open

type:performance

Difficulty 4/5 3-5 days Newbie friendliness 25/100

google/grain#851 · 3 comments · 2 reactions · Opened May 5, 2025
Grain workers allocates GPU memory Open

type:performance

google/grain#820 · 1 comment · 2 reactions · 1 assignee · Opened Apr 18, 2025
osx cant open .so array_ Closed

type:bug

Difficulty 3/5 1-2 days Newbie friendliness 25/100

google/grain#812 · 2 comments · Opened Apr 13, 2025
Amount of time spent accessing thread locks for ArrayRecord datasources Open

type:performance

Difficulty 4/5 3-5 days Newbie friendliness 35/100

google/grain#808 · 1 comment · 1 reaction · Opened Apr 9, 2025
Windows support Closed

type:feature

Difficulty 5/5 Over a week Newbie friendliness 10/100

google/grain#793 · 1 comment · Opened Mar 24, 2025
Correct way to consume multiple files Closed

type:support

Difficulty 5/5 Over a week Newbie friendliness 25/100

google/grain#708 · 1 comment · Opened Jan 26, 2025
error: could not create 'build/lib.macosx-11.0-arm64-cpython-312/grain': Not a directory Open

type:build/install

Difficulty 3/5 1-2 days Newbie friendliness 35/100

google/grain#694 · Opened Jan 16, 2025
comparing array_record's ArrayRecordDataSource + grain data loader to huggingface. (Grain is extremely slow) Open

type:performance

Difficulty 4/5 3-5 days Newbie friendliness 25/100

google/grain#688 · 3 reactions · Opened Jan 9, 2025
Build from source succeeds but subsequent `import`s fail - also ERROR: Configuration file not found: grain/third_party/py/grain/oss/copy.bara.sky Open

type:bug

Difficulty 4/5 3-5 days Newbie friendliness 35/100

google/grain#669 · 4 comments · Opened Dec 21, 2024
How would one create a data source for video? Open

type:support

Difficulty 5/5 Over a week Newbie friendliness 15/100

google/grain#633 · Opened Nov 22, 2024
Nested Dataset Slowdown Open

type:performance

Difficulty 4/5 3-5 days Newbie friendliness 35/100

google/grain#593 · 2 comments · Opened Oct 3, 2024
Slow compared to simple loader Open

type:performance

Difficulty 4/5 3-5 days Newbie friendliness 25/100

google/grain#569 · 5 comments · 1 reaction · Opened Sep 1, 2024
Advice on using a JIT function inside a __getitem__ or transform? Open

type:performance

Difficulty 4/5 3-5 days Newbie friendliness 25/100

google/grain#495 · 3 comments · 1 reaction · Opened Jul 10, 2024
Batching into shared memory is deprecated, but essential for performance Open

type:performance

Difficulty 4/5 3-5 days Newbie friendliness 35/100

google/grain#492 · 3 comments · Opened Jun 28, 2024
whether to support streaming dataset？ Open

type:feature

Difficulty 5/5 Over a week Newbie friendliness 20/100

google/grain#487 · 4 comments · Opened Jun 27, 2024
AttributeError: 'NoneType' object has no attribute 'mmap' Open

type:bug

Difficulty 3/5 1-2 days Newbie friendliness 45/100

google/grain#398 · 4 comments · 6 reactions · Opened Apr 24, 2024
Simple usage example? (eg. MNIST) Closed

type:support

Difficulty 3/5 1-2 days Newbie friendliness 25/100

google/grain#381 · 6 comments · 13 reactions · Opened Apr 6, 2024
Slow data loading of large arrays from sharded dataset Open

type:feature

Difficulty 4/5 3-5 days Newbie friendliness 35/100

google/grain#372 · 3 comments · Opened Mar 30, 2024
error Closed

type:support

google/grain#333 · 2 comments · 1 assignee · Opened Jan 31, 2024

google/grain

Get new issues in your inbox