Add tool-call capability validation to model selection pipeline

Problem

OpenClaw relies heavily on tool/function calling to execute skills — 12+ embedded skills ship with executable scripts (kube.py, rpc.sh, signer.py, buy.py, discovery.py, monetize.py, etc.) that the agent invokes via shell execution tools. If the assigned model doesn't support tool calling, every executable skill silently becomes dead weight. The user gets no warning — the agent appears to work but never invokes any skill.

Root causes

1. Zero capability metadata. ProviderInfo, ProviderStatus, ModelEntry — none carry a supports_tools, supports_vision, or any capability field. The codebase has no concept of model capabilities.

2. ValidateCustomEndpoint() is a ping test only. It sends {role:"user", content:"ping"} with max_tokens:1. It never tests whether the model can produce a tool_calls response. A model that passes validation may be completely incapable of using tools.

3. drop_params: true is the silent killer. Set globally in the LiteLLM ConfigMap (litellm_settings.drop_params: true). When a model doesn't support the tools[] parameter, LiteLLM silently strips it instead of erroring. OpenClaw sends tool definitions → LiteLLM drops them → the model never sees them → skills never fire → no error is raised.

4. rankModels() uses string heuristics, not capabilities. isCloudModel() checks if the name contains "claude" or starts with "gpt", "o1", "o3". That's it. A tiny Ollama model with no tool support can become the primary agent model with no warning.

5. Selling side has zero capability exposure. The ServiceOffer CRD only carries model.name and model.runtime. Buyers paying via x402 have no way to know if the model supports tools, vision, structured output, etc.

Impact

Scenario	What happens today
User sets up Ollama model without tool support	Model is accepted, assigned to OpenClaw. All 12+ executable skills silently fail.
User adds a custom endpoint	ValidateCustomEndpoint passes with any model that can chat. No tool-call check.
Buyer purchases inference via x402	No capability metadata in ServiceOffer. Buyer discovers limitations only after paying.
rankModels() picks primary model	Ranking is cloud-vs-local by name pattern. A cloud model without tool support beats a local model with it.

Inspiration

ToolCall-15 is a focused benchmark that tests 5 categories of LLM tool-calling failures:

• Tool Selection — pick the right tool among 12
• Parameter Precision — pass correct arguments, handle types/units
• Multi-Step Chains — thread data across 4-step dependent calls
• Restraint/Refusal — know when NOT to call a tool
• Error Recovery — handle empty results and failures gracefully

We don't need to implement the full benchmark, but we can borrow its simplest probe pattern: send a tool definition + a message that requires it, and check whether the model produces a valid tool_calls response.

Proposed solution

Phase 1: Tool-call probe during model setup

During obol model setup, after the existing reachability/inference ping, add a tool-call probe:

{
  "model": "<model>",
  "messages": [{"role": "user", "content": "What is the weather in Berlin?"}],
  "tools": [{
    "type": "function",
    "function": {
      "name": "get_weather",
      "description": "Get weather for a location",
      "parameters": {
        "type": "object",
        "properties": {"location": {"type": "string"}},
        "required": ["location"]
      }
    }
  }],
  "max_tokens": 100
}

Pass criteria: Response contains choices[0].message.tool_calls with at least one entry where function.name == "get_weather".

If the model fails:

• Display a clear warning: ⚠ Model "<name>" does not support tool calling. OpenClaw skills will not work.
• Still allow setup (the model may be useful for non-agent workloads) but flag it in model metadata.

Phase 2: Add capability fields to model types

Extend ModelEntry or introduce a new ModelCapabilities struct:

type ModelCapabilities struct {
    SupportsTools          bool `json:"supports_tools" yaml:"supports_tools"`
    SupportsVision         bool `json:"supports_vision" yaml:"supports_vision"`
    SupportsReasoning      bool `json:"supports_reasoning" yaml:"supports_reasoning"`
    SupportsParallelTools  bool `json:"supports_parallel_tools" yaml:"supports_parallel_tools"`
    SupportsStructuredOutput bool `json:"supports_structured_output" yaml:"supports_structured_output"`
}

Populate from:

1. The Phase 1 probe result (definitive for SupportsTools)
2. LiteLLM's GET /model/info endpoint (returns supports_function_calling, supports_tool_choice, supports_vision, supports_reasoning, etc. for 2500+ models from its built-in model cost map)
3. User override via CLI flags

Phase 3: Capability-aware model selection for OpenClaw

• rankModels() should filter to tool-call-capable models first, then apply the cloud-vs-local ranking within that set.
• If no tool-capable model is available, warn loudly at startup and during obol openclaw operations.
• Consider gating OpenClaw deployment: if zero models pass the tool-call probe, block deployment with a clear error rather than deploying a broken agent.

Phase 4: Expose capabilities on the selling side

• Add capability fields to the ServiceOffer CRD spec (alongside model.name and model.runtime).
• Populate them from the probe/LiteLLM metadata when obol monetize registers the service.
• Surface capabilities in the inference gateway response headers or a discovery endpoint.
• Wire into ERC-8004 registration metadata (the services[].Skills OASF taxonomy and registration.metadata map already support this — they're just unused for capability info today).

LiteLLM /model/info reference

LiteLLM already tracks per-model capabilities in its model cost database (2594 models, 1353 with supports_function_calling). The proxy exposes this via GET /model/info:

{
  "data": [{
    "model_name": "claude-sonnet-4-20250514",
    "model_info": {
      "supports_function_calling": true,
      "supports_tool_choice": true,
      "supports_parallel_function_calling": true,
      "supports_vision": true,
      "supports_reasoning": true,
      "supports_response_schema": true,
      "max_input_tokens": 200000,
      "max_output_tokens": 64000
    }
  }]
}

Today obol-stack only queries /v1/models (bare model IDs). Switching to /model/info would give us capability metadata for free on cloud models.

Files to change

File	Change
internal/model/model.go	Add tool-call probe to ValidateCustomEndpoint(), add ModelCapabilities struct, query /model/info in queryLiteLLMModels()
internal/openclaw/openclaw.go	Update rankModels() to use capabilities, add warnings in SyncOverlayModels()
internal/inference/gateway.go	Expose capability metadata in response headers or discovery endpoint
internal/schemas/serviceoffer.go	Add capability fields to ServiceOffer spec
internal/embed/infrastructure/base/templates/llm.yaml	Consider documenting/commenting the drop_params: true risk

Start by tracing model setup and model discovery in internal/model/model.go, especially ValidateCustomEndpoint() and queryLiteLLMModels(). Then follow rankModels() in internal/openclaw/openclaw.go and the ServiceOffer and gateway types in the listed files. Done means tool-call capability is recorded, considered during model selection, and exposed or warned about across the proposed setup and selling flows.