Inferred response body for 4xx response in MinimalAPI app is "TaskOfTodo"

### Is there an existing issue for this?

- [X] I have searched the existing issues

### Describe the bug

In a minimal API app I have the following endpoint:
```csharp
app.MapGet("/body-schema-inferred",
[ProducesResponseType(200)]
[ProducesResponseType(404)]
[ProducesResponseType(500)]
async Task (int id, TodoDb db) =>
await db.Todos.FindAsync(id));
```

In the generated OpenAPI, the 200 response is rendered as expected:
```json
"200": {
"description": "OK",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/Todo"
}
}
}
},
```

but the 404 and 500 responses have a whacky schema `TaskOfTodo`
```json
"404": {
"description": "Not Found",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/TaskOfTodo"
}
}
}
},
"500": {
"description": "Internal Server Error",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/TaskOfTodo"
}
}
}
}
```

which is defined in the `components.schemas` section as:
```json
"TaskOfTodo": {
"type": "object",
"properties": {
"result": {
"$ref": "#/components/schemas/Todo2"
},
"id": {
"type": "integer",
"format": "int32"
},
"exception": {
"$ref": "#/components/schemas/AggregateException"
},
"status": {
"$ref": "#/components/schemas/TaskStatus"
},
"isCanceled": {
"type": "boolean"
},
"isCompleted": {
"type": "boolean"
},
"isCompletedSuccessfully": {
"type": "boolean"
},
"creationOptions": {
"$ref": "#/components/schemas/TaskCreationOptions"
},
"asyncState": { },
"isFaulted": {
"type": "boolean"
}
}
},
```

### Expected Behavior

In a controller-based app, 4xx responses that are defined by ProducesResponseType get a default schema of ProblemDetails, and no schema is inferred for non-4xx status codes. I think this would be the most desirable resolution.

A next best option would be to not default the schema of a non-2xx response -- leave it as unspecified.

### Steps To Reproduce

I can provide a repro if needed.

### Exceptions (if any)

_No response_

### .NET Version

9.0.100-rc.1.24452.12

### Anything else?

_No response_