LangGraph adapter: agents with declared outputs fail on Cohere models (ToolStrategy forces tool_choice, which langchain-oci rejects for Cohere)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 420
- Forks
- 60
- Avg merge
- 9h 30m
- Merged PRs (30d)
- 4
Description
Description
In the LangGraph adapter, any Agent that declares outputs fails on its first model call when the LLM is a Cohere model on OCI Generative AI (OciGenAiConfig with e.g. cohere.command-a-03-2025):
ValueError: Tool choice is not supported for Cohere models.Please remove the tool_choice parameter.
The same agent without outputs works, with or without tools. Found while investigating #224 and #227 (the structured-output path of the adapter).
Reproduction
from pyagentspec.adapters.langgraph import AgentSpecLoader
from pyagentspec.agent import Agent
from pyagentspec.llms import OciGenAiConfig
from pyagentspec.llms.ociclientconfig import OciClientConfigWithSecurityToken
from pyagentspec.property import StringProperty
llm = OciGenAiConfig(
name="oci",
model_id="cohere.command-a-03-2025",
compartment_id="<compartment_ocid>",
client_config=OciClientConfigWithSecurityToken(
name="cc",
service_endpoint="https://inference.generativeai.<region>.oci.oraclecloud.com",
auth_file_location="~/.oci/config",
auth_profile="<profile>",
),
)
agent = Agent(
name="assistant",
llm_config=llm,
system_prompt="You are a concise assistant.",
outputs=[StringProperty(title="answer")], # <- remove this line and the agent works
)
graph = AgentSpecLoader().load_component(agent) # OK
graph.invoke({"messages": [{"role": "user", "content": "What is 3 times 6?"}]},
{"configurable": {"thread_id": "1"}}) # fails
Observed live with langchain-oci 0.2.6, langchain 1.3.9, langgraph 1.2.4, Python 3.12:
| agent | result |
|---|---|
| no outputs, no tools | works, answers 18 |
| a tool, no outputs | works, calls the tool |
outputs=[answer], no tools |
ValueError at first invoke |
outputs=[answer] + a tool |
ValueError at first invoke |
Traceback excerpt:
File ".../langchain/agents/factory.py", line 1364, in _get_bound_model
request.model.bind_tools(
File ".../langchain_oci/chat_models/oci_generative_ai.py", line 311, in bind_tools
kwargs["tool_choice"] = self._provider.process_tool_choice(tool_choice)
File ".../langchain_oci/chat_models/providers/cohere.py", line 721, in process_tool_choice
raise ValueError("Tool choice is not supported for Cohere models."
"Please remove the tool_choice parameter.")
Root cause
_create_react_agent_with_given_info(pyagentspec/src/pyagentspec/adapters/langgraph/_langgraphconverter.py, around theresponse_format = ToolStrategy(output_model)line) always usesToolStrategyfor agents with outputs, regardless of the provider.- LangChain's
create_agentbinds the structured-output tool withtool_choice="any"whenever aToolStrategyis active (langchain/agents/factory.py,_get_bound_model:tool_choice = "any" if structured_output_tools else request.tool_choice). - langchain-oci's
CohereProvider.process_tool_choiceraises for any non-Nonetool_choice. This is unchanged in langchain-oci 0.3.2 (the latest release) and on itsmain, so the dependency bump in #250 does not change the outcome.
The failure only surfaces at the first model call, not when loading the component.
Expected behavior
Agents with declared outputs run on Cohere models, or the adapter reports a clear configuration error at load time.
Suggested fixes
- For providers that do not support
tool_choice(Cohere viaChatOCIGenAI), do not useToolStrategy; use the provider-native structured output instead.ChatOCIGenAI.with_structured_outputsupportsmethod="json_schema"for Cohere (CohereProvider.oci_json_schema_response_format), which maps to LangChain'sProviderStrategy. - Alternatively, keep
ToolStrategybut bind the structured-output tool without forcingtool_choicefor such providers (needs a LangChain-side hook or a middleware adjusting the model request). - At minimum, detect the combination at load time and raise a configuration error naming the model, the
outputs, and the workaround (declare a single string output or use another model), instead of the provider's low-level error at run time.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in pyagentspec/src/pyagentspec/adapters/langgraph/_langgraphconverter.py, especially _create_react_agent_with_given_info around the response_format = ToolStrategy(output_model) line, and reproduce the first-invoke failure with the Cohere OciGenAiConfig example. The work is done when an agent with declared outputs either runs on Cohere models or reports a clear configuration error at load time, with adapter coverage updated as appropriate.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api, backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 66/100