mlcommons/storage

View on GitHub

MLPerf® Storage Benchmark Suite

Stars: 203
Forks: 67
Open beginner issues: 1
Indexed issues: 69
Avg merge: 20m
Merged PRs (30d): 8

Dominant language: Python
License: Apache-2.0
Last GitHub push: Sep 20, 2026
Latest indexed: Sep 19, 2026
Contributing guide: Contributing guide
Code of conduct: No code of conduct
Beginner labels: No beginner labels indexed

Beginner friendly Everything

Open All

69 open issues indexed

Running KVcache after VectorDB causes a hang in cluster_collector.py Open

Difficulty 4/5 3-5 days Newbie friendliness 45/100

mlcommons/storage#843 · 1 comment · Opened Aug 11, 2026
Six submission-checker rules never fire on real trees: verification compared lowercase-only against uppercase metadata Open

mlpstorage

Difficulty 4/5 3-5 days Newbie friendliness 62/100

mlcommons/storage#842 · Opened Aug 10, 2026
Rules.md 4.3.5: a missing "not" inverted subset-run validation — subset mode is enforced backwards Open

mlpstorage

Difficulty 4/5 3-5 days Newbie friendliness 28/100

mlcommons/storage#841 · 1 comment · Opened Aug 10, 2026
Finite replay cycles drop queued requests Open

Future KVCache TF

mlcommons/storage#839 · 1 assignee · Opened Aug 2, 2026
Rules.md does not describe public_ids.json, which reportgen now writes into submission packages Open

Future mlpstorage Rules

Difficulty 4/5 3-5 days Newbie friendliness 45/100

mlcommons/storage#838 · Opened Jul 30, 2026
reportgen: non-canonical kv_cache run layout publishes one run's metrics and silently discards the rest Open

Future mlpstorage

Difficulty 4/5 3-5 days Newbie friendliness 48/100

mlcommons/storage#836 · 1 comment · Opened Jul 29, 2026
reportgen silently drops kv_cache/vdb runs missing *_metadata.json — metrics publish as blank Open

Future mlpstorage

Difficulty 3/5 1-2 days Newbie friendliness 55/100

mlcommons/storage#835 · 1 comment · Opened Jul 28, 2026
Test fixtures for training & checkpointing use fabricated DLIO metric keys real DLIO never emits Open

Future mlpstorage

mlcommons/storage#830 · 1 assignee · Opened Jul 21, 2026
KV cache: seeded runs produce different logical file-I/O workloads Open

Future KVCache TF

Difficulty 5/5 Over a week Newbie friendliness 45/100

mlcommons/storage#822 · 7 comments · Opened Jul 20, 2026
[Bug] MLPStorage Validation Fails with "Expected 6 run files, but found 12" for RetinaNet on Both B200 and MI355 Open

mlcommons/storage#679 · 5 comments · 1 assignee · Opened Jul 5, 2026
DLIO summary.json: host_memory_GB / host_cpu_count arrays are per-nnodes-slot, not per-host — positional access is broken Open

DLIO or mlpstorage Future

mlcommons/storage#676 · 1 assignee · Opened Jul 5, 2026
Clarification on client host memory requirement for 1T model checkpointing – changed between v2.0 and v3.0? Open

documentation Rules

Difficulty 3/5 1-2 days Newbie friendliness 48/100

mlcommons/storage#518 · 3 comments · Opened Jun 25, 2026
^C doesn't abort all the worker processes Open

Future

mlcommons/storage#515 · 1 assignee · Opened Jun 25, 2026
Enforce --data-dir post-YAML for all training commands; eliminate inherited DLIO YAML default Open

Future

Difficulty 4/5 3-5 days Newbie friendliness 74/100

mlcommons/storage#508 · 1 comment · Opened Jun 24, 2026
Intermittent errors between epochs Open

bug DLIO or mlpstorage Future

Difficulty 4/5 3-5 days Newbie friendliness 35/100

mlcommons/storage#499 · 2 comments · Opened Jun 23, 2026
DISKANN 10M×1536-dim recall requires search_list=4000 to reach 95%, causing unacceptable latency and throughput Open

VectorDB TF

mlcommons/storage#425 · 6 comments · 1 assignee · Opened Jun 10, 2026
KVCache: eviction-driven tier I/O is excluded from latency P95 SLA checks (Storage/CPU write & read latencies never recorded in _demote_entry) Open

Future KVCache TF

Difficulty 3/5 1-2 days Newbie friendliness 72/100

mlcommons/storage#421 · 2 comments · 1 reaction · Opened Jun 9, 2026
[Training 3.0] Add interlock file to data directories Open

enhancement Training

Difficulty 4/5 3-5 days Newbie friendliness 45/100

mlcommons/storage#390 · Opened May 26, 2026
[Training v3.0 consolidation] Ensure the Parquet data generator delivers reasonable generation throughput Open

enhancement Future PASS Score Blocker Training

Difficulty 5/5 Over a week Newbie friendliness 35/100

mlcommons/storage#358 · Opened Apr 27, 2026
[Training v3.0 consolidation] Evaluate Parquet reader performance in FLUX to validate it can meet target throughput Open

enhancement Future PASS Score Blocker Training

Difficulty 4/5 3-5 days Newbie friendliness 35/100

mlcommons/storage#357 · Opened Apr 27, 2026
[Training v3.0 consolidation] Keep a single Parquet file handle open in the FLUX reader to avoid row group cache eviction Open

enhancement Future PASS Score Blocker Training

Difficulty 4/5 3-5 days Newbie friendliness 48/100

mlcommons/storage#356 · Opened Apr 27, 2026
[Training v3.0 consolidation] Evaluate Arrow IPC as alternative format for DLRM Open

enhancement Future PASS Score Blocker Training

Difficulty 5/5 Over a week Newbie friendliness 25/100

mlcommons/storage#355 · Opened Apr 27, 2026
[Training v3.0 consolidation] DLRM optimization: evaluate Parquet reader and generation parameters to improve throughput Open

enhancement Future PASS Score Blocker Training

Difficulty 5/5 Over a week Newbie friendliness 35/100

mlcommons/storage#354 · 5 comments · Opened Apr 27, 2026
[Training v3.0 consolidation] Confirm computation time step for DLRM Open

bug Future PASS Score Blocker Training

Difficulty 3/5 1-2 days Newbie friendliness 48/100

mlcommons/storage#353 · Opened Apr 27, 2026
ValueError: 'parquet' is not a valid FormatType when using mlpstorage training datagen Open

Future

Difficulty 3/5 1-2 days Newbie friendliness 45/100

mlcommons/storage#336 · Opened Apr 15, 2026
KVCache Verify client CPU choice has minimal influence on results Open

Future KVCache TF

Difficulty 4/5 3-5 days Newbie friendliness 48/100

mlcommons/storage#334 · 1 comment · Opened Apr 14, 2026
[Proposal] Evaluate Arrow IPC format for reader Open

enhancement Future

Difficulty 5/5 Over a week Newbie friendliness 35/100

mlcommons/storage#333 · 3 comments · Opened Apr 14, 2026
mlpstorage error with proxy, needs httpx[socks] Open

enhancement Future

Difficulty 3/5 1-2 days Newbie friendliness 45/100

mlcommons/storage#332 · 3 comments · Opened Apr 14, 2026
mlpstorage training run --model=flux has considerably lower I/O Throughput causing train_au_meet_expectation to fail Open

bug Future

Difficulty 4/5 3-5 days Newbie friendliness 45/100

mlcommons/storage#330 · 6 comments · Opened Apr 12, 2026
dlio_s3torch_checkpoint fails with TLS (SSL) negotiation failed errors Open

Difficulty 3/5 1-2 days Newbie friendliness 48/100

mlcommons/storage#327 · 1 comment · Opened Apr 9, 2026
Add S3 config parameters to flux/dlrm/retinanet yaml files Open

Difficulty 3/5 1-2 days Newbie friendliness 68/100

mlcommons/storage#326 · 2 comments · Opened Apr 9, 2026
workload yaml files with hard coded endpoint_url resulting in DLIO workloads to S3 storage fail Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

mlcommons/storage#325 · 5 comments · Opened Apr 8, 2026
[Passing score blocker] `ParquetReader` involves a CPU time in `get_sample` 5x / 4x times longer than the I/O time Open

bug Future

Difficulty 4/5 3-5 days Newbie friendliness 45/100

mlcommons/storage#323 · 1 comment · Opened Apr 8, 2026
How do we create the python env on worker nodes? Open

DLIO or mlpstorage

Difficulty 4/5 3-5 days Newbie friendliness 35/100

mlcommons/storage#309 · Opened Apr 3, 2026
Add Object parameters to all the workload yaml files Open

enhancement

Difficulty 3/5 1-2 days Newbie friendliness 55/100

mlcommons/storage#291 · 1 comment · Opened Mar 26, 2026
NPZ CRC Checksum overhead for File I/O - Expansion of issue #223 Open

Difficulty 5/5 Over a week Newbie friendliness 45/100

mlcommons/storage#286 · 1 reaction · Opened Mar 23, 2026
KV-Cache does not support Object Storage Open

Future KVCache TF

Difficulty 4/5 3-5 days Newbie friendliness 48/100

mlcommons/storage#285 · 3 comments · Opened Mar 23, 2026
KVCache: Prefix Cache Implementation Does Not Match Design Specification Open

enhancement Future KVCache TF

mlcommons/storage#283 · 3 comments · 1 assignee · Opened Mar 20, 2026
retinanet runs forever without generating I/O Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

mlcommons/storage#258 · 1 comment · Opened Mar 3, 2026
Turn off the CRC check in DLIO's use of numpy.load() Open

mlcommons/storage#223 · 1 assignee · Opened Jan 9, 2026
KVBench - Add support for different random distributions (random, uniform, zipfian) when selecting chunks for RAG usecase Open

enhancement KVCache TF

Difficulty 4/5 3-5 days Newbie friendliness 48/100

mlcommons/storage#216 · 2 comments · Opened Nov 25, 2025
mlpstorage arguments have confusing names that conflict with IEC standards Open

Difficulty 2/5 1-3 hours Newbie friendliness 62/100

mlcommons/storage#207 · 1 comment · Opened Oct 17, 2025
RockyLinux / RHEL support? Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

mlcommons/storage#206 · Opened Oct 16, 2025
The problem that the reader.read_thread parameter does not utilize as much as the actual number of nproc. Open

Difficulty 4/5 3-5 days Newbie friendliness 38/100

mlcommons/storage#205 · 1 comment · Opened Oct 13, 2025
What it mean ? : [WARNING] Running DLIO with 32 threads for I/O but core available 2 are insufficient and can lead to lower performance. Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

mlcommons/storage#202 · 7 comments · Opened Sep 23, 2025
I was wondering if the cpu-bind option is released or if there are cases where the numa-core allocation may not be done properly. Open

Difficulty 4/5 3-5 days Newbie friendliness 20/100

mlcommons/storage#201 · 1 comment · Opened Sep 16, 2025
Error creating dataset with resnet50/cosmoflow using multiple hosts Open

Difficulty 3/5 1-2 days Newbie friendliness 45/100

mlcommons/storage#200 · Opened Sep 15, 2025
mlpstorage checkpointing datasize run failed Open

Difficulty 2/5 1-3 hours Newbie friendliness 35/100

mlcommons/storage#198 · 2 comments · Opened Sep 8, 2025
PMIX ERR NOMEM error, but the test can still run Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

mlcommons/storage#197 · 3 comments · Opened Sep 1, 2025
`--exec-type=docker` Launches Only a Single Process, Ignores `--num-accelerators` Open

Difficulty 3/5 1-2 days Newbie friendliness 50/100

mlcommons/storage#196 · 1 comment · Opened Aug 12, 2025

mlcommons/storage

Get new issues in your inbox