tile-ai / tile-ai/TileRT

Issue: Failed to load libtilert.so due to undefined symbol _ZN3c104cuda29c10_cuda_check_implementationEiPKcS2_ib

Open
#6 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.8k
Forks
124
PR merge metrics
No merged PRs in 30d

Description

Issue: Failed to load libtilert.so due to undefined symbol _ZN3c104cuda29c10_cuda_check_implementationEiPKcS2_ib

Description

When running the generate script in a Docker environment, I encounter an error loading the libtilert.so library due to an undefined symbol related to PyTorch's CUDA checks.

Error Log

(tilert) [root@d9878fa39ee8 workspace]# python generate.py     --model-weights-dir "/home/model_weights" 2>&1 | tee generate.log
Traceback (most recent call last):
  File "/opt/conda/envs/tilert/lib/python3.12/site-packages/tilert/__init__.py", line 38, in _load_library
    return ctypes.CDLL(str(lib_path))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/tilert/lib/python3.12/ctypes/__init__.py", line 379, in __init__
    self._handle = _dlopen(self._name, mode)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: /opt/conda/envs/tilert/lib/python3.12/site-packages/tilert/libtilert.so: undefined symbol: _ZN3c104cuda29c10_cuda_check_implementationEiPKcS2_ib

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workspace/generate.py", line 9, in <module>
    from tilert import logger, tilert_init
  File "/opt/conda/envs/tilert/lib/python3.12/site-packages/tilert/__init__.py", line 43, in <module>
    _load_library("libtilert.so")
  File "/opt/conda/envs/tilert/lib/python3.12/site-packages/tilert/__init__.py", line 40, in _load_library
    raise RuntimeError(f"Failed to load library from {lib_path}") from e
RuntimeError: Failed to load library from /opt/conda/envs/tilert/lib/python3.12/site-packages/tilert/libtilert.so

Environment

  • Running in Docker container

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the failure with generate.py in the Docker environment, then inspect tilert/init.py where libtilert.so is loaded and compare the installed PyTorch/CUDA environment with the library's requirements. Done means generate.py loads libtilert.so without the undefined-symbol error and proceeds past the import.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, python, pytorch
Domain
devops, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.