huggingface / huggingface/inference-benchmarker

k8s deployment failure tgi libpython3.11.so.1.0: cannot open shared object file: No such file or directory

Open
#31 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
172
Forks
34
PR merge metrics
No merged PRs in 30d

Description

Deploying the inference-benchmarker on an OpenShift 4.18 cluster fails with [OSError: PermissionError](libpython3.11.so.1.0: cannot open shared object file: No such file or directory).

I am using the helm chart with the following `extra/k8s/inference-benchmarker/values.yaml`:

```yaml
...
model_id: "mistralai/Mistral-7B-Instruct-v0.3"
server: tgi
....
tgi:
enabled: true
```

I enabled my HF token to access the gated model 'mistralai/Mistral-7B-Instruct-v0.3'.

```bash
$ helm install inference-benchmarker ./extra/k8s/inference-benchmarker
NAME: inference-benchmarker
LAST DEPLOYED: Mon Jul 7 10:42:47 2025
NAMESPACE: blue
STATUS: deployed
REVISION: 1
TEST SUITE: None
```

This is the output of the OpenShift Pod "inference-benchmarker-799b8587-hdbfz" CrashLoopBackOff
```bash
unable to refresh ld cache, not a big deal in most cases
text-generation-launcher: error while loading shared libraries: libpython3.11.so.1.0: cannot open shared object file: No such file or directory
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.