mlcommons / mlcommons/endpoints-submission-cli
Separate 'run date' and 'run upload date'
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1
- Forks
- 1
- Avg merge
- 2d 13h
- Merged PRs (30d)
- 6
Description
Summary
The submission CLI currently conflates two distinct concepts under a single "date": the date a benchmark run actually happened, and the date a run/submission was uploaded to the API. We should treat these as two separate fields with different semantics.
Background
Today, runs create and submissions create-local each compute a single timestamp for a run in different, inconsistent ways (one derives it from the benchmark's own log when available and falls back to "now"; the other always uses "now"). That timestamp is then used to populate the run_date field, which can end up not reflecting when the benchmark was actually executed.
Proposal
Introduce two clearly separate fields:
- Run date — the date the benchmark run actually started/completed. This must come from the benchmark's own output log (e.g.
result_summary.json). It should never be inferred or defaulted by the CLI.- If the run log does not contain this information, the CLI should error out rather than silently substituting a fallback value (like the current upload time). This is a signal that the benchmark output itself is missing required data.
- Run upload date — the date the run/submission was uploaded via the CLI. This can simply be the current timestamp at upload time, since that's genuinely what it represents.
Notes
- The exact field/schema for how "run date" is emitted by the benchmark log is still being finalized, so this issue is to track the requirement, not to implement it yet.
- Once the benchmark output format is finalized, the CLI's run-parsing logic should be updated to read run date strictly from the log and fail fast if it's absent, and to record upload date as a separate, independent field.
This issue was created by Claude.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the runs create and submissions create-local command paths and their run-parsing logic. Wait for the benchmark output schema to be finalized, then verify that run date is read only from the log, upload date is recorded independently, and missing run-date data produces an error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100