posit-dev / posit-dev/ai-lib

Per-protocol capability profiles: overriding a model's protocol should re-derive inferred capabilities

Open
#62 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
5
Forks
2
Avg merge
13h 33m
Merged PRs (30d)
35

Description

Follow-up to #61 (Databricks native APIs), which documents this limitation in memory-bank/aiConfig.md.

Problem

Protocol-stamping providers (LiteLLM, and now Databricks) infer a model's capabilities for the protocol they stamp. A natively-routed Databricks Claude model advertises thinkingEffortLevels and PDF input because the anthropic-messages route supports them; the same endpoint forced onto openai-chat (via a user protocol override or a connection-level protocol) does not support those capabilities — but it keeps the stamped ones.

The generic fix ("mask capabilities when final protocol ≠ stamp") was considered and withdrawn during planning: resolveModels merges user capability overrides into the flat model object before attachRouting, so a pipeline mask cannot distinguish user-supplied capability values from inferred ones, and no safe protocol-keyed mask exists post-merge.

Proposed direction

Introduce a provenance/profile seam in the resolution pipeline: carry inferred capabilities as a per-protocol profile (or tag capability values with their provenance — inferred vs user override) so that when the final resolved protocol differs from the discovery stamp, inferred capabilities can be re-derived for the actual route while user overrides still win.

Scope notes

  • Pre-existing limitation for LiteLLM; Databricks native routing widens the surface.
  • Related: a profile-key transport through chat params (separate issue) could share the same seam.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read memory-bank/aiConfig.md, then trace the resolveModels and attachRouting stages described in the issue. Determine how inferred capabilities can retain protocol provenance while user overrides remain authoritative; done means a protocol override re-derives stamped capabilities for the final route without discarding explicit user values.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
ai, backend-api-design
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.