create: empty mutable-field registry breaks --set for many nouns (all AI-Evals, environment, service, repository) with no -f fallback
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 22
- Forks
- 10
- Avg merge
- 14h 28m
- Merged PRs (30d)
- 54
Description
Summary
For many create nouns, the --set key=value flag rejects every field with unknown or read-only field "<field>", even though --help advertises those exact fields. --list-fields confirms the cause: these nouns return "No mutable fields defined for this command" — the mutable-field registry is empty, so no --set value is ever accepted. There is also no -f/--file fallback, leaving these nouns with no working create path via the CLI at all.
This appears to be the same root cause as #27 (connector) and #23 (PR), but the scope is much broader — it affects all AI-Evals nouns plus several core ones.
Version
harness version 0.1.10 (2026-06-18T04:38:52Z)
Reproduction
$ harness create eval_target --org default --project <proj> \
--set name=repro --set identifier=repro --set type=prompt
unknown or read-only field "type"; use --list-fields to see mutable fields
$ harness create eval_target --list-fields
No mutable fields defined for this command.
--help for the same command advertises the fields it then rejects:
$ harness create eval_target --help
Create an AI Evals target: harness create eval_target --set name=... identifier=... type=prompt
Scope — --list-fields audit
Working (registry populated):
| Noun | --list-fields |
|---|---|
secret |
ID / Label / Type ✅ |
project |
ID / Label / Type ✅ |
organization |
ID / Label / Type ✅ |
Broken (empty registry → --set unusable, no -f fallback):
| Noun | --list-fields |
|---|---|
eval_target |
No mutable fields ❌ |
eval_dataset |
No mutable fields ❌ |
eval_metric |
No mutable fields ❌ |
eval_model |
No mutable fields ❌ |
eval_metric_set |
No mutable fields ❌ |
evaluation |
No mutable fields ❌ |
pr |
No mutable fields ❌ (see #23) |
connector |
No mutable fields ❌ (see #27) |
environment |
No mutable fields ❌ |
service |
No mutable fields ❌ |
repository |
No mutable fields ❌ |
No workaround
The broken nouns also reject -f:
$ harness create eval_target -f /dev/null
unknown shorthand flag: 'f' in -f
So affected resources cannot be created through the CLI by any means; callers must drop to the REST API.
Expected
Either:
- Populate the mutable-field registry for these nouns so the
--setfields shown in--helpactually work (and support nested keys likegit_source.file_pathfor git-backed eval resources), or - Add a
-f/--file <yaml|json>body flag as a general create path for complex/nested resources.
Impact
Standing up an AI-Evals stack (dataset → target → metric set → evaluation) or any git-backed eval is impossible via the CLI today — every component create fails. Same for code pr, connector, environment, service, and repository create.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the create command's --set, --list-fields, and -f/--file handling, comparing the mutable-field registry for working nouns with eval_target and the other listed broken nouns. Done means the advertised fields are accepted for affected resources, including nested git_source.file_path where applicable, or a general file-based create path is available.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- cli, developer-experience
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100