openai / openai/openai-openapi
GraderMulti.graders models one grader instead of a named dictionary
Nobody has claimed this yet.
- Dominant language
- No language data
- Stars
- 2.5k
- Forks
- 527
- Avg merge
- 1h 46m
- Merged PRs (30d)
- 2
Description
Source report
Mirrors openai-java#647, which reports a generated MultiGrader request failing with 400: Invalid type for 'graders.input': expected an object, but got a string instead.
Specification mismatch
In the current public specification, GraderMulti.graders is a oneOf containing individual StringCheck, TextSimilarity, Python, ScoreModel, and LabelModel graders. It therefore describes a single grader object.
The combined-graders guide instead defines graders as an object whose keys name constituent graders. calculate_output refers to those keys. For example:
{
"type": "multi",
"name": "example",
"graders": {
"compliant": {
"type": "string_check",
"name": "compliant",
"input": "{{sample.output_json.compliant}}",
"reference": "{{item.compliant}}",
"operation": "eq"
}
},
"calculate_output": "0.5 * compliant"
}
The generated SDK currently follows the single-grader union, so the source example serializes the grader fields directly under graders and the service treats input as a named grader with an invalid string value.
There is also an inconsistent GraderMulti example in the specification: it uses an array, which matches neither the single-grader schema nor the documented named-map contract.
Expected correction
Describe graders as an object whose additional properties use the supported individual-grader union, and replace the array example with a named map whose keys match calculate_output. Regenerate the published schema and SDK models. Preserve the original SDK reproduction as a validation case.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with openapi.yaml around line 41033 and compare the GraderMulti schema with the combined-graders guide. Inspect the inconsistent GraderMulti example and the generated schema and SDK models. Done means the named-map contract and matching example are represented, regeneration completes, and the original SDK reproduction validates the correction.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- openapi
- Domain
- api
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100