"Invalid State: Controller is already closed" error when trying to use chat-ui with trained llama2 model on the HF platform

When I try to run the llama2 model card that I trained, with chat-ui in Space using Nvidia A10G small, I get no response.

I see the following error in the container log:

"""
07:43:00 3|index | TypeError [ERR_INVALID_STATE]: Invalid state: Controller is already closed
07:43:00 3|index | at new NodeError (node:internal/errors:399:5)
07:43:00 3|index | at ReadableStreamDefaultController.enqueue (node:internal/webstreams/readablestream:1036:13)
07:43:00 3|index | at update (file:///app/build/server/chunks/_server.ts-2f619497.js:451:20)
07:43:00 3|index | at file:///app/build/server/chunks/_server.ts-2f619497.js:461:13
07:43:00 3|index | at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
07:43:00 3|index | at async Object.start (file:///app/build/server/chunks/_server.ts-2f619497.js:552:7) {
07:43:00 3|index | code: 'ERR_INVALID_STATE'
07:43:00 3|index | }
"""

Chat UI files:

env.local.template file used:
```python
# Use .env.local to change these variables
# DO NOT EDIT THIS FILE WITH SENSITIVE DATA

MONGODB_URL=${MONGODB_URL}
MONGODB_DB_NAME=chat-ui
MONGODB_DIRECT_CONNECTION=false

COOKIE_NAME=chat-ui
HF_ACCESS_TOKEN=#hf_ from from https://huggingface.co/settings/token

# used to activate search with web functionality. disabled if not defined
SERPAPI_KEY=#your serpapi key here

# Parameters to enable "Sign in with HF"
OPENID_CLIENT_ID=
OPENID_CLIENT_SECRET=
OPENID_SCOPES="openid profile" # Add "email" for some providers like Google that do not provide preferred_username
OPENID_PROVIDER_URL=https://huggingface.co # for Google, use https://accounts.google.com

# 'name', 'userMessageToken', 'assistantMessageToken' are required
MODELS=`[
{
"name": "${MODEL_NAME}",
"chatPromptTemplate": "${MODEL_PROMPT_TEMPLATE}",
"preprompt": "",
"promptExamples": [
{
"title": "Python Fibonacci",
"prompt": "How can I write a Python function to generate the nth Fibonacci number?"
}, {
"title": "What is a meme?",
"prompt": "What is a meme, and what's the history behind this word?"
}, {
"title": "Regex",
"prompt": "Create a regex to extract dates from logs"
}
],
"endpoints": [
{
"type": "tgi",
"url": "http://127.0.0.1:8080"
}
],
"parameters": ${MODEL_PARAMS}
}
]`
OLD_MODELS=`[]`# any removed models, `{ name: string, displayName?: string, id?: string }`

PUBLIC_ORIGIN=${SPACE_HOST}
PUBLIC_SHARE_PREFIX=${SPACE_HOST}/r
PUBLIC_GOOGLE_ANALYTICS_ID=#G-XXXXXXXX / Leave empty to disable
PUBLIC_DEPRECATED_GOOGLE_ANALYTICS_ID=#UA-XXXXXXXX-X / Leave empty to disable
PUBLIC_ANNOUNCEMENT_BANNERS=`[
{
"title": "Chat UI is now open sourced on GitHub",
"linkTitle": "GitHub repo",
"linkHref": "https://github.com/huggingface/chat-ui"
}
]`

PARQUET_EXPORT_DATASET=
PARQUET_EXPORT_HF_TOKEN=
PARQUET_EXPORT_SECRET=

PUBLIC_APP_NAME=${APP_NAME} # name used as title throughout the app
PUBLIC_APP_ASSETS=chatui # used to find logos & favicons in static/$PUBLIC_APP_ASSETS
PUBLIC_APP_COLOR=${APP_COLOR} # can be any of tailwind colors: https://tailwindcss.com/docs/customizing-colors#default-color-palette
PUBLIC_APP_DATA_SHARING=#set to 1 to enable disclaimers & options about data sharing
PUBLIC_APP_DATA_DISCLAIMER=#set to 1 to enable disclaimers about model outputs
```

entrypoint.sh.template used:
```python
#!/bin/bash

# Start the local Mongo database
mongod &

# Start the text-generation-inference process
text-generation-launcher --model-id ${MODEL_NAME} --num-shard 1 --port 8080 --trust-remote-code &

# Wait for text-generation-inference to start
curl --retry 60 --retry-delay 10 --retry-connrefused http://127.0.0.1:8080/health

# Start the chat-ui process
pm2 start /app/build/index.js -i $CPU_CORES --no-daemon &

# Wait for any process to exit
wait -n

# Exit with status of process that exited first
exit $?
```

The interesting thing is that when I send a one word text, although I can't get a reply from the chat section in the interface, I can see the short reply from the preview in the left menu. I am sharing a screenshot below explaining this situation:

another example for the same situation: (run on Nvidia 10G Large)

Screenshot of the container error I mentioned at the beginning of the Issue:

chat-ui uses "text-generation-launcher" to run the model. People who previously solved the same error for llama.cpp solved the problem by setting the "np" argument to 2. When I looked at the documentation of "text-generation-launcher" I couldn't see a similar argument. I did tests by playing with the VALIDATION_WORKERS, NUM_SHARD arguments here ( https://huggingface.co/docs/text-generation-inference/basic_tutorials/launcher). But I couldn't get any result.

Could there be something I might have missed here? Does anyone have a solution you can suggest on this?

Thanks.