SFTtech / SFTtech/openage

Make fails with python/cython error: undefined reference to pthread_*, sem_*, dl*

Open
#1,435 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug lang: cython
Dominant language
Python
Stars
14.4k
Forks
1.3k
PR merge metrics
No merged PRs in 30d

Description

Hi, I came across this particular issue (that I have resolved[found a workaround for]) in the make stage where what I think are the python modules (cython I imagine), fails to find these respective libraries that it gives an error for.

Environment
  • Ubuntu 20.0.4
  • cmake 3.22.0
  • gcc 10.3.0
  • python 3.6.10 using pyenv (a virtualenv/python installation handler)
Error
./Python/dynload_shlib.c:82: error: undefined reference to 'dlsym'
./Python/dynload_shlib.c:95: error: undefined reference to 'dlopen'
./Python/dynload_shlib.c:126: error: undefined reference to 'dlsym'
./Python/dynload_shlib.c:95: error: undefined reference to 'dlopen'
./Python/dynload_shlib.c:101: error: undefined reference to 'dlerror'
Python/thread_pthread.h:252: error: undefined reference to 'pthread_create'
Python/thread_pthread.h:269: error: undefined reference to 'pthread_detach'
Python/thread_pthread.h:233: error: undefined reference to 'pthread_attr_setstacksize'
Python/thread_pthread.h:322: error: undefined reference to 'sem_init'
Python/thread_pthread.h:347: error: undefined reference to 'sem_destroy'
Python/thread_pthread.h:386: error: undefined reference to 'sem_wait'
Python/thread_pthread.h:386: error: undefined reference to 'sem_wait'
Python/thread_pthread.h:384: error: undefined reference to 'sem_trywait'
Python/thread_pthread.h:382: error: undefined reference to 'sem_timedwait'
Python/thread_pthread.h:428: error: undefined reference to 'sem_post'
Python/thread_pthread.h:649: error: undefined reference to 'pthread_key_create'
Python/thread_pthread.h:654: error: undefined reference to 'pthread_key_delete'
Python/thread_pthread.h:677: error: undefined reference to 'pthread_setspecific'
Python/thread_pthread.h:627: error: undefined reference to 'pthread_attr_setstacksize'
Python/thread_pthread.h:664: error: undefined reference to 'pthread_key_delete'
Python/thread_pthread.h:670: error: undefined reference to 'pthread_setspecific'
Python/thread_pthread.h:684: error: undefined reference to 'pthread_getspecific'
./Modules/posixmodule.c:5921: error: undefined reference to 'forkpty'
./Modules/posixmodule.c:5827: error: undefined reference to 'openpty'
./Modules/signalmodule.c:1119: error: undefined reference to 'pthread_kill'
./Modules/signalmodule.c:855: error: undefined reference to 'pthread_sigmask'
./Modules/faulthandler.c:577: error: undefined reference to 'pthread_sigmask'
collect2: error: ld returned 1 exit status
Workaround/Fix

The issue I think was caused because of me using pyenv for managing my python versions and virtual environments. Pyenv allows me to easily mange and install multiple python versions, and virtual environments, but it therefore does not manage them in the default path. This if not catered for was what led to me believe would cause these issues. Perhaps the problem was that when python is installed using pyenv it does not link against these libraries properly, I could not test this out.
What I tried and managed to do to fix this was to use my systems python installation, by disabling pyenv for that session. That was done using pyenv local system (specific to pyenv).
I don't know if this issue would arise for virtualenv as well since I have not used that, but it did arise for pyenv. I managed to proceed and compile, build and test the project successfully afterwards, so theres definitely something conflicting with how the cmake setup might/might not work with pyenv atleast.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the make-stage linker failure on Ubuntu with the pyenv-managed Python, then compare it with the successful system-Python build. Inspect the CMake setup for Python extension link libraries and verify that the project builds and tests successfully while using pyenv.

Written by the indexing model from the issue text.

Assessment

Tech stack
cmake, python
Domain
build-system
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.