explosion / explosion/spacy-llm

LLM cannot be configured using `langchain.Databricks.v1`

Open
#493 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.4k
Forks
110
PR merge metrics
No merged PRs in 30d

Description

It is not possible to configure `spacy_llm` to instantiate an LLM agent using `langchain.Databricks`.

For example, here is my initial config

```bash
[components.llm.model]
@llm_models = "langchain.Databricks.v1"
name = "databricks-meta-llama-3-70b-instruct"
query = {"@llm_queries": "spacy.CallLangChain.v1"}
```

This gives me the following error indicating that the `endpoint_name` must be set in the config

```python
File ~/****/.venv/lib/python3.11/site-packages/pydantic/main.py:214, in BaseModel.__init__(self, **data)
212 # `__tracebackhide__` tells pytest and some other tools to omit this function from tracebacks
213 __tracebackhide__ = True
--> 214 validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
215 if self is not validated_self:
216 warnings.warn(
217 'A custom validator is returning a value other than `self`.\n'
218 "Returning anything other than `self` from a top level model validator isn't supported when validating via `__init__`.\n"
219 'See the `model_validator` docs (https://docs.pydantic.dev/latest/concepts/validators/#model-validators) for more details.',
220 stacklevel=2,
221 )

ValidationError: 1 validation error for Databricks
Value error, Neither endpoint_name nor cluster_id was set. And the cluster_id cannot be automatically determined. Received error: Cannot access dbruntime, not running inside a Databricks notebook. [type=value_error, input_value={'model': 'databricks-meta-llama-3-70b-instruct'}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.10/v/value_error
```

Modifying the config to include an endpoint name then raises a different validation error because `lanchain_community.llm.databricks.Databricks` does not accept a `name` argument

```bash
[components.llm.model]
@llm_models = "langchain.Databricks.v1"
name = "databricks-meta-llama-3-70b-instruct"
query = {"@llm_queries": "spacy.CallLangChain.v1"}
config = {"endpoint_name": "databricks-meta-llama-3-70b-instruct"}
```

raises

```python
ValidationError: 1 validation error for Databricks
model
Extra inputs are not permitted [type=extra_forbidden, input_value='databricks-meta-llama-3-70b-instruct', input_type=str]
For further information visit https://errors.pydantic.dev/2.10/v/extra_forbidden
```

This API has been [deprecated](https://python.langchain.com/api_reference/community/llms/langchain_community.llms.databricks.Databricks.html) in favour of `langchain_databricks.ChatDatabricks`, so is there a recommended way to interface with LLMs using `spacy_llm` and the Databricks platform?

For reference, these are the packages I am using:

```bash
spacy-llm==0.7.3
langchain==0.3.14
langchain-community==0.3.14
langchain-databricks==0.1.2
databricks-connect==15.4.3
databricks-langchain==0.1.1
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing the registered `langchain.Databricks.v1` model and `spacy.CallLangChain.v1` query, then compare their expected configuration with the installed `langchain-community` and `langchain-databricks` APIs. Confirm whether the deprecated Databricks wrapper can be supported or whether the newer API needs an integration path. Done means a validated configuration or a documented limitation with an appropriate test.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.