huggingface / huggingface/chat-ui
Fail to create assistant.
- Dominant language
- TypeScript
- Stars
- 11k
- Forks
- 1.7k
- Avg merge
- 21h 24m
- Merged PRs (30d)
- 56
Description
I use the docker image chat-ui-db as the frontend, text-generation-inference as the inference backend, and meta-llamaLlama-2-70b-chat-hf as the model. Using the image and model mentioned above, I set up a large language model dialog service on server A. Assume that the IP address of the server A is x.x.x.x.
I use docker compose to deploy it. The content of docker-compose.yml is as follows:
```
services:
chat-ui:
image: chat-ui-db:latest
ports:
- "3000:3000"
restart: unless-stopped
textgen:
image: huggingface/text-generation-inference:1.4
ports:
- "8080:80"
command: ["--model-id", "/data/models/meta-llamaLlama-2-70b-chat-hf"]
volumes:
- /home/test/llm-test/serving/data:/data
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: 8
capabilities: [gpu]
restart: unless-stopped
```
I set ENABLE_ASSISTANTS=true in .env.local to enable assistants feature.
I logged into localhost:3000 using chrome, clicked the settings button, and then clicked the create new assistant button. Enter the information in the Name and Description text boxes, select a model, and enter the information in the User start messages and Instructions (system prompt) text boxes. Finally, click the Create button. I can create an assistant just fine.
When I go to xxxx:3000 from a browser on a different server and access the service. (One may ask, how can I achieve access to server A's services from other servers without logging. The solution is to use nginx as a http to https anti-proxy(https://www.inovex.de/de/blog/code-assistant-how-to-self-host-your-own/)). I clicked the settings button, and then clicked the create new assistant button. Enter the information in the Name and Description text boxes, select a model, and enter the information in the User start messages and Instructions (system prompt) text boxes. Finally, click the Create button. The webpage is not responding. The container logs don't show anything either. I couldn't create an assistant.
What should i do?
Do I have to enable login authentication to create an assistant? unless I'm accessing it from localhost. I'm on a LAN and I can't get user authentication through Huggingface or google. I have also tried to set up a user authentication service using keycloak and configure .env.local to enable open id login. But the attempt failed. See this page(https://github.com/huggingface/chat-ui/issues/896) for the specific problem.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.