mlcommons / mlcommons/mlcube_cookiecutter

Proposal: scaffold optional audit manifest example

Open Beginner friendly
#10 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
0
Forks
2
PR merge metrics
No merged PRs in 30d

Description

Proposal

Would this cookiecutter template be open to scaffolding an optional audit/provenance manifest example for generated MLCube projects?

This is related to the broader MLCube proposal here: https://github.com/mlcommons/mlcube/issues/367

The cookiecutter may be the safest place to start because it can teach good reproducibility/audit hygiene without changing MLCube runtime behavior.

Suggested generated file

For example, generated MLCubes could include an optional file such as:

{{cookiecutter.mlcube_name}}/
  audit_manifest.example.json

Example contents:

{
  "schema_version": "mlcube.run_audit.v1",
  "mlcube_name": "{{cookiecutter.mlcube_name}}",
  "task": "{{cookiecutter.mlcube_name}}",
  "runner": "docker",
  "image": "mlcommons/{{cookiecutter.mlcube_name}}:0.0.1",
  "inputs": ["workspace/parameters_file.yaml"],
  "outputs": ["workspace/output.txt"],
  "claim_status": "diagnostic",
  "redaction_status": "safe_for_public_log"
}

Why this may help

  • gives new MLCube authors a reproducibility/provenance pattern from the first generated project
  • creates a standard place to document run inputs, outputs, runner, image, and claim status
  • encourages audit-safe result publishing without storing raw secrets or sensitive arguments
  • stays optional and does not affect existing runner behavior

Possible first PR scope

If maintainers think this is useful, I can prepare a small PR that either:

  1. adds audit_manifest.example.json to the generated template, or
  2. keeps it docs-only in the README if that is preferred

This is motivated by AANA work around audit-safe AI evaluation artifacts, but the contribution would be generic to MLCube and would not require AANA as a dependency.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by inspecting the generated template and the README to determine whether the optional example belongs in the scaffold or documentation. Use the proposed audit_manifest.example.json contents and confirm that existing runner behavior remains unchanged; done means the example is generated or documented without requiring AANA or runtime changes.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
tooling
Issue type
Feature
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.