huggingface / huggingface/aisheets
[Feature request] Enable API token for custom `MODEL_ENDPOINT_URL`
- Dominant language
- TypeScript
- Stars
- 1.6k
- Forks
- 140
- Avg merge
- 1m
- Merged PRs (30d)
- 1
Description
If `MODEL_ENDPOINT_URL=http://litellm:4000/` is set but the OpenAI-compatible endpoint expects an API key, there's no way to insert it except to use the `HF_TOKEN`. This would work, however it disables other authentication points to HF services, giving errors like
> ❌ [buildWebSearchQueries] Error generating search queries: InferenceClientProviderApiError: Failed to perform inference: Couldn't authenticate. Reason: Unable authenticate
at innerRequest (file:///usr/src/app/node_modules/.pnpm/@huggingface+inference@4.4.0/node_modules/@huggingface/inference/dist/esm/utils/request.js:45:23)
at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
at async chatCompletion (file:///usr/src/app/node_modules/.pnpm/@huggingface+inference@4.4.0/node_modules/@huggingface/inference/dist/esm/tasks/nlp/chatCompletion.js:10:32)
at async yl (file:///usr/src/app/server/@qwik-city-plan.js:250:358)
at async zp (file:///usr/src/app/server/@qwik-city-plan.js:251:452)
at async qp (file:///usr/src/app/server/@qwik-city-plan.js:235:15488)
at async Hp (file:///usr/src/app/server/@qwik-city-plan.js:235:16045)
at async Qp (file:///usr/src/app/server/@qwik-city-plan.js:235:16471)
at async Wt (file:///usr/src/app/server/@qwik-city-plan.js:235:13059)
at async Object. (file:///usr/src/app/server/@qwik-city-plan.js:251:1728) {
httpRequest: {
url: 'https://api.studio.nebius.ai/v1/chat/completions',
method: 'POST',
headers: {
Authorization: 'Bearer [redacted]',
'Content-Type': 'application/json',
'User-Agent': '@huggingface/inference/4.4.0 Node.js/22'
},
body: {
messages: [Array],
model: 'meta-llama/Llama-3.3-70B-Instruct-fast'
}
},
httpResponse: {
requestId: '5ebfe1aefb7967f2da2107db0db6d16c',
status: 401,
body: { detail: "Couldn't authenticate. Reason: Unable authenticate" }
}
}
A good solution would be to add a `MODEL_ENDPOINT_API_TOKEN` env var
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by tracing where MODEL_ENDPOINT_URL and HF_TOKEN are read and passed into the OpenAI-compatible and Hugging Face inference paths. Add support for a separate MODEL_ENDPOINT_API_TOKEN, then verify custom endpoint authentication works without changing authentication for HF services.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- node.js, typescript
- Domain
- api, backend
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100