huggingface / huggingface/aisheets

[Feature request] Enable API token for custom `MODEL_ENDPOINT_URL`

Open
#472 0 comments 2 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
1.6k
Forks
140
Avg merge
1m
Merged PRs (30d)
1

Description

If `MODEL_ENDPOINT_URL=http://litellm:4000/` is set but the OpenAI-compatible endpoint expects an API key, there's no way to insert it except to use the `HF_TOKEN`. This would work, however it disables other authentication points to HF services, giving errors like

> ❌ [buildWebSearchQueries] Error generating search queries: InferenceClientProviderApiError: Failed to perform inference: Couldn't authenticate. Reason: Unable authenticate
at innerRequest (file:///usr/src/app/node_modules/.pnpm/@huggingface+inference@4.4.0/node_modules/@huggingface/inference/dist/esm/utils/request.js:45:23)
at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
at async chatCompletion (file:///usr/src/app/node_modules/.pnpm/@huggingface+inference@4.4.0/node_modules/@huggingface/inference/dist/esm/tasks/nlp/chatCompletion.js:10:32)
at async yl (file:///usr/src/app/server/@qwik-city-plan.js:250:358)
at async zp (file:///usr/src/app/server/@qwik-city-plan.js:251:452)
at async qp (file:///usr/src/app/server/@qwik-city-plan.js:235:15488)
at async Hp (file:///usr/src/app/server/@qwik-city-plan.js:235:16045)
at async Qp (file:///usr/src/app/server/@qwik-city-plan.js:235:16471)
at async Wt (file:///usr/src/app/server/@qwik-city-plan.js:235:13059)
at async Object. (file:///usr/src/app/server/@qwik-city-plan.js:251:1728) {
httpRequest: {
url: 'https://api.studio.nebius.ai/v1/chat/completions',
method: 'POST',
headers: {
Authorization: 'Bearer [redacted]',
'Content-Type': 'application/json',
'User-Agent': '@huggingface/inference/4.4.0 Node.js/22'
},
body: {
messages: [Array],
model: 'meta-llama/Llama-3.3-70B-Instruct-fast'
}
},
httpResponse: {
requestId: '5ebfe1aefb7967f2da2107db0db6d16c',
status: 401,
body: { detail: "Couldn't authenticate. Reason: Unable authenticate" }
}
}

A good solution would be to add a `MODEL_ENDPOINT_API_TOKEN` env var

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing where MODEL_ENDPOINT_URL and HF_TOKEN are read and passed into the OpenAI-compatible and Hugging Face inference paths. Add support for a separate MODEL_ENDPOINT_API_TOKEN, then verify custom endpoint authentication works without changing authentication for HF services.

Written by the indexing model from the issue text.

Assessment

Tech stack
node.js, typescript
Domain
api, backend
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.