OpenAiApi's ChatCompletion class needs more flexibility to be fully compatible with other providers

Bug description
See here https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-openai/src/main/java/org/springframework/ai/openai/api/OpenAiApi.java#L1752.

Due to the restrictive nature of the ChatCompletion class, several OpenAI-compatible providers that use Spring AI's OpenAiApi will end up losing important fields in the response object that is ultimately returned by the ChatResponse.

For example, Perplexity AI returns its citations and search results in their own respsective objects at the top level of the response, which are ultimately ignored and removed by the time the ChatResponse is returned.

xAI/Grok behaves similarly and its web search results are also not included in the response due to this limitation.

Environment
Spring AI 1.1.0

Steps to reproduce
Attempt to send a query to Perplexity AI's Sonar via Spring AI, and inspect the response object that is returned by ChatResponse.

Expected behavior
Prompt: What were the results of the 2025 French Open Finals?

{
    "results": [
        {
            "output": {
                "messageType": "ASSISTANT",
                "metadata": {
                    "role": "ASSISTANT",
                    "messageType": "ASSISTANT",
                    "refusal": "",
                    "finishReason": "STOP",
                    "annotations": [
                        {}
                    ],
                    "index": 0,
                    "id": "80705ade-d12d-44f3-8495-63ebb76b2d24"
                },
                "toolCalls": [],
                "media": [],
                "text": "**Carlos Alcaraz won the 2025 French Open men's singles final against Jannik Sinner with a score of 4–6, 6–7(4–7), 6–4, 7–6(7–3), 7–6(10–2), marking his second straight French Open title and fifth major overall.[1][4][5][6]**\n\nAlcaraz staged a comeback from two sets down, saving three championship points in an epic 5-hour-29-minute match—the longest French Open final in history and second-longest major final ever—which ended Sinner's perfect record in major finals.[1][2][4]\n\n### Key Match Details\n- **Set breakdown**: Sinner took the first two sets; Alcaraz won the next three in tiebreaks after both players served for the championship but were broken.[1]\n- **Historic notes**: Alcaraz became the third man in the Open Era to win a major after facing championship points (after Gaudio in 2004 French Open and Djokovic in 2019 Wimbledon); he improved his head-to-head vs. Sinner to 8-4.[1][2]\n\n### Women's Singles Final\nSearch results do not provide the final score or winner; one source mentions Barbora Krejčíková as a winner at Roland Garros but lacks specifics on the singles final result.[2]\n\n### Other Notable Results\n- **Men's doubles**: Marcel Granollers and Horacio Zeballos won the title.[2]\n- **Top seeds (men's)**: #1 Jannik Sinner (finalist), #2 Carlos Alcaraz (champion), #6 Novak Djokovic (semifinals).[1]"
            },
            "metadata": {
                "finishReason": "STOP",
                "contentFilters": [],
                "empty": true
            }
        }
    ],
    "result": {
        "output": {
            "messageType": "ASSISTANT",
            "metadata": {
                "role": "ASSISTANT",
                "messageType": "ASSISTANT",
                "refusal": "",
                "finishReason": "STOP",
                "annotations": [
                    {}
                ],
                "index": 0,
                "id": "80705ade-d12d-44f3-8495-63ebb76b2d24"
            },
            "toolCalls": [],
            "media": [],
            "text": "**Carlos Alcaraz won the 2025 French Open men's singles final against Jannik Sinner with a score of 4–6, 6–7(4–7), 6–4, 7–6(7–3), 7–6(10–2), marking his second straight French Open title and fifth major overall.[1][4][5][6]**\n\nAlcaraz staged a comeback from two sets down, saving three championship points in an epic 5-hour-29-minute match—the longest French Open final in history and second-longest major final ever—which ended Sinner's perfect record in major finals.[1][2][4]\n\n### Key Match Details\n- **Set breakdown**: Sinner took the first two sets; Alcaraz won the next three in tiebreaks after both players served for the championship but were broken.[1]\n- **Historic notes**: Alcaraz became the third man in the Open Era to win a major after facing championship points (after Gaudio in 2004 French Open and Djokovic in 2019 Wimbledon); he improved his head-to-head vs. Sinner to 8-4.[1][2]\n\n### Women's Singles Final\nSearch results do not provide the final score or winner; one source mentions Barbora Krejčíková as a winner at Roland Garros but lacks specifics on the singles final result.[2]\n\n### Other Notable Results\n- **Men's doubles**: Marcel Granollers and Horacio Zeballos won the title.[2]\n- **Top seeds (men's)**: #1 Jannik Sinner (finalist), #2 Carlos Alcaraz (champion), #6 Novak Djokovic (semifinals).[1]"
        },
        "metadata": {
            "finishReason": "STOP",
            "contentFilters": [],
            "empty": true
        }
    },
    "metadata": {
        "id": "80705ade-d12d-44f3-8495-63ebb76b2d24",
        "model": "sonar",
        "rateLimit": {},
        "usage": {
            "promptTokens": 13,
            "completionTokens": 364,
            "totalTokens": 377,
            "nativeUsage": {
                "completion_tokens": 364,
                "prompt_tokens": 13,
                "total_tokens": 377
            }
        },
        "promptMetadata": [],
        "empty": false
    }
}

Notice that the annotations object remains empty, because Perplexity returns citations in a different field at the top level (not the message level), taken from Perplexity docs (https://docs.perplexity.ai/getting-started/quickstart#heres-an-example-response-raw-response-at-the-end):

{
  "id": "66f3900f-e32e-4d59-b677-1a55de188262",
  "model": "sonar-pro",
  "created": 1756485272,
  "usage": {
    "prompt_tokens": 12,
    "completion_tokens": 315,
    "total_tokens": 327,
    "search_context_size": "low",
    "cost": {
      "input_tokens_cost": 0.0,
      "output_tokens_cost": 0.005,
      "request_cost": 0.006,
      "total_cost": 0.011
    }
  },
  "citations": [
    "https://en.wikipedia.org/wiki/2025_French_Open_%E2%80%93_Men's_singles_final",
    "https://www.espn.com/tennis/scoreboard/tournament/_/eventId/172-2025/competitionType/1",
    "https://www.cbssports.com/tennis/news/2025-french-open-results-schedule-as-jannik-sinner-faces-carlos-alcaraz-coco-gauff-earns-first-title/",
    "https://www.youtube.com/watch?v=jrkwqoI-gEg",
    "https://en.wikipedia.org/wiki/2025_French_Open_%E2%80%93_Men's_singles"
  ],
  "search_results": [
    {
      "title": "2025 French Open – Men's singles final",
      "url": "https://en.wikipedia.org/wiki/2025_French_Open_%E2%80%93_Men's_singles_final",
      "date": "2025-06-08",
      "last_updated": "2025-08-09",
      "snippet": "After 5 hours and 29 minutes of play, Alcaraz defeated Sinner 4–6, 6–7 (4–7) , 6–4, 7–6 (7–3) , 7–6 (10–2) , in the longest French Open final in history."
    },
    {
      "title": "2025 Roland Garros Men's Singles Tennis Live Scores - ESPN",
      "url": "https://www.espn.com/tennis/scoreboard/tournament/_/eventId/172-2025/competitionType/1",
      "date": "2025-06-08",
      "last_updated": "2025-08-29",
      "snippet": "2025 Roland Garros Scores May 18 - June 8, 2025 Court Philippe-Chatrier, Paris, France Men's Singles 2025 Carlos Alcaraz Defending Champion Carlos Alcaraz."
    },
    {
      "title": "2025 French Open: Results, schedule as Jannik Sinner ...",
      "url": "https://www.cbssports.com/tennis/news/2025-french-open-results-schedule-as-jannik-sinner-faces-carlos-alcaraz-coco-gauff-earns-first-title/",
      "date": "2025-06-07",
      "last_updated": "2025-08-29",
      "snippet": "The women's final is on June 7, and the men's final is one day later on June 8. Men's final. (1) Jannik Sinner vs. (2) Carlos Alcaraz -- Sunday, ..."
    },
    {
      "title": "Alcaraz, Gauff Win French Open 2025 | Swiatek, Ruud Fall - YouTube",
      "url": "https://www.youtube.com/watch?v=jrkwqoI-gEg",
      "date": "2025-06-09",
      "last_updated": "2025-08-04",
      "snippet": "We had some big changes in the rankings also. What was your favorite moment? 0:00 | Intro 0:14 | Weekly Results 0:36 | Rise & Fall 1:24 ..."
    },
    {
      "title": "2025 French Open – Men's singles",
      "url": "https://en.wikipedia.org/wiki/2025_French_Open_%E2%80%93_Men's_singles",
      "date": "2025-04-27",
      "last_updated": "2025-08-29",
      "snippet": "Defending champion Carlos Alcaraz defeated Jannik Sinner in the final, 4–6, 6–7, 6–4, 7–6, 7–6 to win the men's singles tennis title at the 2025 French Open. ..."
    }
  ],
  "object": "chat.completion",
  "choices": [
    {
      "index": 0,
      "finish_reason": "stop",
      "message": {
        "role": "assistant",
        "content": "**Carlos Alcaraz won the 2025 French Open men's singles final, defeating Jannik Sinner 4–6, 6–7(4–7), 6–4, 7–6(7–3), 7–6(10–2), while Coco Gauff won the women's singles title by rallying past Aryna Sabalenka in three sets**[1][3][5][4].\n\nKey details from the finals:\n\n- **Men's Singles:**  \n  - Alcaraz came back from two sets down to win in the longest French Open final ever (5 hours, 29 minutes)[1][5].\n  - He saved three championship points, a record in the Open Era for men's majors[1][5].\n  - This marked his second Roland Garros title and fifth Grand Slam overall[5].\n  - The match was the first French Open singles final decided by a match tiebreak (final set tiebreak introduced in 2022)[1][5].\n\n- **Women's Singles:**  \n  - Coco Gauff defeated Aryna Sabalenka after losing the first set, showcasing a strong comeback[3].\n  - Gauff secured her second Grand Slam (her first was at the 2023 US Open)[3].\n  - The final was played June 7, 2025; Gauff overcame an early deficit to win in three sets[3].\n\nThese finals were historic for their drama, length, and the milestone achievements for both Alcaraz and Gauff."
      },
      "delta": {
        "role": "assistant",
        "content": ""
      }
    }
  ]
}

Next steps
A change is required to the ChatCompletion class such that extra fields that aren't exactly matching with OpenAI's completion definition are not excluded, or for support to be made similarly to how extra_body is handled with JsonAnySetter such that there is no need to hardcode any new fields while still truly supporting other OpenAI-compatible providers.

In its current state, it cannot be really said that Spring AI's OpenAI implementation supports OpenAI-compatible server when the response object is incomplete/missing key fields.

I wonder if it's possible to do something similar to the official OpenAI Java SDK, as seen here:

/** Represents a chat completion response returned by model, based on the provided input. */
class ChatCompletion
@JsonCreator(mode = JsonCreator.Mode.DISABLED)
private constructor(
    private val id: JsonField<String>,
    private val choices: JsonField<List<Choice>>,
    private val created: JsonField<Long>,
    private val model: JsonField<String>,
    private val object_: JsonValue,
    private val serviceTier: JsonField<ServiceTier>,
    private val systemFingerprint: JsonField<String>,
    private val usage: JsonField<CompletionUsage>,
    private val additionalProperties: MutableMap<String, JsonValue>,
) {

See more here: https://github.com/openai/openai-java/blob/main/openai-java-core/src/main/kotlin/com/openai/models/chat/completions/ChatCompletion.kt#L25

Start in models/spring-ai-openai/src/main/java/org/springframework/ai/openai/api/OpenAiApi.java at the ChatCompletion class around line 1752. Compare its response deserialization with the cited OpenAI Java SDK model and determine how unknown top-level fields such as Perplexity's citations and search_results can be retained. Done means ChatResponse no longer loses provider-specific response fields while preserving existing OpenAI fields.