mlcommons / mlcommons/endpoints

schema.py: dedupe cyclopts help= vs Field(description=) across config fields

Open
#451 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
21
Forks
28
Avg merge
3d 17h
Merged PRs (30d)
13

Description

Follow-up from PR #406 review: https://github.com/mlcommons/endpoints/pull/406#discussion_r3799230536

Background

Several config fields in src/inference_endpoint/config/schema.py set the same help text twice — once as cyclopts.Parameter(help=...) and once as Field(description=...). This is redundant:

  • cyclopts (4.10.0) reads Field(description=...) natively for --help. When both are set, help= wins; delete help= and cyclopts falls back to descriptionidentical CLI output.
  • scripts/regenerate_templates.py (_collect_comments._help) already reads description first, help only as fallback.

So Field(description=...) can be the single source of truth for both the CLI --help text and the YAML template comments.

Proposal

Make Field(description=...) canonical and drop the redundant help=. Keep cyclopts.Parameter(...) only where it carries CLI-only metadata (alias, negative, name, show, required) — those have no Pydantic equivalent. No decorator/machinery needed: cyclopts already does the injection, so this is less machinery, not more.

(Avoid the bare attribute-docstring form — pydantic doesn't put it in FieldInfo.description, so the template generator would lose the comment.)

Scope (as of HEAD 6d28c00)

16 fields set both help + description10 identical (safe to collapse), 6 silently divergent; plus 17 help-only and 50 description-only (already single-source).

Identical — just drop help=
  • ModelParams.seed
  • WarmupConfig.{enabled, n_requests, drain, warmup_random_seed}salt already collapsed in #406
  • DrainConfig.{warmup_timeout_s, performance_timeout_s, accuracy_timeout_s}
  • ProfilingConfig.engine
⚠️ Silently divergent — CLI --help and the YAML comment already say different things

Single-sourcing prevents exactly this drift; decide per field which text is correct before collapsing.

field help= (CLI) Field(description=) (YAML/schema)
EndpointConfig.endpoints Endpoint URL(s) Endpoint URL(s). Must include scheme, e.g. 'http://host:port'.
EarlyStoppingConfig.enabled Report MLPerf early-stopping percentile estimates for TTFT/TPOT/latency Early-stopping percentile estimates (default on)
Settings.service_ready_timeout_s Seconds to wait for metrics/event-logger services to start Seconds to wait for metrics-aggregator/event-logger services to become ready.
ProfilingConfig.urls Override URL(s) for profiler triggers; defaults to endpoint_config.endpoints URL(s) the profiler start/stop triggers are derived from. When None, derived from endpoint_config.endpoints instead…
DrainConfig.metrics_drain_timeout_s …Set to 0 to wait indefinitely. Increase for very large datasets… …after ENDED (default: 0 = unlimited). An incomplete drain is surfaced via n_pending_tasks > 0, never silently dropped.
DrainConfig.metrics_tokenizer_workers …0 defers all tokenization to the end-of-run drain, which always uses the auto-sized sharded pool. …(default: 2; 0 = defer everything to the end-of-run drain).

Already done

PR #406 collapsed only WarmupConfig.salt (that PR edited its help text in both places, so it was in scope). This issue tracks the remaining 15 fields.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in src/inference_endpoint/config/schema.py and review PR #406 for the existing collapse pattern. Compare each listed cyclopts help value with its Field description, resolve the six divergent cases, and remove redundant help values while preserving CLI-only metadata. Run scripts/regenerate_templates.py and verify that CLI help and YAML template comments retain the intended text.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli, documentation
Issue type
Refactor
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.