opensearch-project / opensearch-project/flow-framework
[FEATURE] Support internal connector in register model
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 62
- Forks
- 66
- Avg merge
- 3d 20h
- Merged PRs (30d)
- 20
Description
Is your feature request related to a problem?
Right now we have a step called RegisterRemoteModel in our framework that correlates to the register_model API In ml-commons when a remote model is given. Currently we only support creating a remote model with a given connector ID. For example:
POST /_plugins/_ml/models/_register
{
"name": "openAI-gpt-3.5-turbo",
"function_name": "remote",
"model_group_id": "1jriBYsBq7EKuKzZX131",
"description": "test model",
"connector_id": "a1eMb4kBJ1eYAeTMAljY"
}
However, ml-commons also supports creating an internal connector inside the model itself through this API. We currently don't support this, example payload:
POST /_plugins/_ml/models/_register
{
"name": "openAI-GPT-3.5: internal connector",
"function_name": "remote",
"model_group_id": "lEFGL4kB4ubqQRzegPo2",
"description": "test model",
"connector": {
"name": "OpenAI Connector",
"description": "The connector to public OpenAI model service for GPT 3.5",
"version": 1,
"protocol": "http",
"parameters": {
"endpoint": "api.openai.com",
"max_tokens": 7,
"temperature": 0,
"model": "text-davinci-003"
},
"credential": {
"openAI_key": "..."
},
"actions": [
{
"action_type": "predict",
"method": "POST",
"url": "https://${parameters.endpoint}/v1/completions",
"headers": {
"Authorization": "Bearer ${credential.openAI_key}"
},
"request_body": "{ \"model\": \"${parameters.model}\", \"prompt\": \"${parameters.prompt}\", \"max_tokens\": ${parameters.max_tokens}, \"temperature\": ${parameters.temperature} }"
}
]
}
}
What solution would you like?
There can be two solutions for this, either we create a new step name called register_model_with_internal_connector/register_internal_connector/register_internal_model. Or we change the register_remote_model step to allow for either a connector ID or a connector object by changing the parsing method here: https://github.com/opensearch-project/flow-framework/blob/main/src/main/java/org/opensearch/flowframework/workflow/RegisterRemoteModelStep.java#L109
To satisfy what a connector object is, we can potentially refactor some of the code in the CreateConnectorStep so we don't have duplicates for the connector parsing.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the parsing logic in src/main/java/org/opensearch/flowframework/workflow/RegisterRemoteModelStep.java near line 109, then compare connector handling in CreateConnectorStep.java. Resolve whether the existing step or a new step should accept an inline connector object, and verify that the model registration payload supports the documented internal-connector example without duplicated parsing logic.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- ai, backend-api-design
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100