oracle / oracle/agent-spec

LangGraph adapter: agents with declared outputs fail on Cohere models (ToolStrategy forces tool_choice, which langchain-oci rejects for Cohere)

Open
#262 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
420
Forks
60
Avg merge
9h 30m
Merged PRs (30d)
4

Description

Description

In the LangGraph adapter, any Agent that declares outputs fails on its first model call when the LLM is a Cohere model on OCI Generative AI (OciGenAiConfig with e.g. cohere.command-a-03-2025):

ValueError: Tool choice is not supported for Cohere models.Please remove the tool_choice parameter.

The same agent without outputs works, with or without tools. Found while investigating #224 and #227 (the structured-output path of the adapter).

Reproduction

from pyagentspec.adapters.langgraph import AgentSpecLoader
from pyagentspec.agent import Agent
from pyagentspec.llms import OciGenAiConfig
from pyagentspec.llms.ociclientconfig import OciClientConfigWithSecurityToken
from pyagentspec.property import StringProperty

llm = OciGenAiConfig(
    name="oci",
    model_id="cohere.command-a-03-2025",
    compartment_id="<compartment_ocid>",
    client_config=OciClientConfigWithSecurityToken(
        name="cc",
        service_endpoint="https://inference.generativeai.<region>.oci.oraclecloud.com",
        auth_file_location="~/.oci/config",
        auth_profile="<profile>",
    ),
)
agent = Agent(
    name="assistant",
    llm_config=llm,
    system_prompt="You are a concise assistant.",
    outputs=[StringProperty(title="answer")],   # <- remove this line and the agent works
)
graph = AgentSpecLoader().load_component(agent)   # OK
graph.invoke({"messages": [{"role": "user", "content": "What is 3 times 6?"}]},
             {"configurable": {"thread_id": "1"}})  # fails

Observed live with langchain-oci 0.2.6, langchain 1.3.9, langgraph 1.2.4, Python 3.12:

agent result
no outputs, no tools works, answers 18
a tool, no outputs works, calls the tool
outputs=[answer], no tools ValueError at first invoke
outputs=[answer] + a tool ValueError at first invoke

Traceback excerpt:

  File ".../langchain/agents/factory.py", line 1364, in _get_bound_model
    request.model.bind_tools(
  File ".../langchain_oci/chat_models/oci_generative_ai.py", line 311, in bind_tools
    kwargs["tool_choice"] = self._provider.process_tool_choice(tool_choice)
  File ".../langchain_oci/chat_models/providers/cohere.py", line 721, in process_tool_choice
    raise ValueError("Tool choice is not supported for Cohere models."
                     "Please remove the tool_choice parameter.")

Root cause

  • _create_react_agent_with_given_info (pyagentspec/src/pyagentspec/adapters/langgraph/_langgraphconverter.py, around the response_format = ToolStrategy(output_model) line) always uses ToolStrategy for agents with outputs, regardless of the provider.
  • LangChain's create_agent binds the structured-output tool with tool_choice="any" whenever a ToolStrategy is active (langchain/agents/factory.py, _get_bound_model: tool_choice = "any" if structured_output_tools else request.tool_choice).
  • langchain-oci's CohereProvider.process_tool_choice raises for any non-None tool_choice. This is unchanged in langchain-oci 0.3.2 (the latest release) and on its main, so the dependency bump in #250 does not change the outcome.

The failure only surfaces at the first model call, not when loading the component.

Expected behavior

Agents with declared outputs run on Cohere models, or the adapter reports a clear configuration error at load time.

Suggested fixes

  1. For providers that do not support tool_choice (Cohere via ChatOCIGenAI), do not use ToolStrategy; use the provider-native structured output instead. ChatOCIGenAI.with_structured_output supports method="json_schema" for Cohere (CohereProvider.oci_json_schema_response_format), which maps to LangChain's ProviderStrategy.
  2. Alternatively, keep ToolStrategy but bind the structured-output tool without forcing tool_choice for such providers (needs a LangChain-side hook or a middleware adjusting the model request).
  3. At minimum, detect the combination at load time and raise a configuration error naming the model, the outputs, and the workaround (declare a single string output or use another model), instead of the provider's low-level error at run time.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in pyagentspec/src/pyagentspec/adapters/langgraph/_langgraphconverter.py, especially _create_react_agent_with_given_info around the response_format = ToolStrategy(output_model) line, and reproduce the first-invoke failure with the Cohere OciGenAiConfig example. The work is done when an agent with declared outputs either runs on Cohere models or reports a clear configuration error at load time, with adapter coverage updated as appropriate.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
66/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.