eclipse-cyclonedds / eclipse-cyclonedds/cyclonedds-python

Question on building self-contained wheels

Open
#283 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
110
Forks
68
Avg merge
1h 8m
Merged PRs (30d)
1

Description

Hi, is building a standalone wheel a supported feature, or something that is reserved for CI builds? It would be a very appreciated feature to have, but when I try

STANDALONE_WHEELS=1 pip wheel --no-deps .

the wheel build succeeds, but attempting to use it fails with:

tonic@funbox:~/src/cyclonedds-python/ex$ python -c 'from cyclonedds.tools.wheel_idlc import command; command()' -l py asdf.idl
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/tonic/src/cyclonedds-python/ex/cyclonedds/__init__.py", line 13, in <module>
    from . import internal, util, qos, core, domain, topic, pub, sub, builtin, dynamic, idl
  File "/home/tonic/src/cyclonedds-python/ex/cyclonedds/internal.py", line 25, in <module>
    from .__library__ import library_path, in_wheel
  File "/home/tonic/src/cyclonedds-python/ex/cyclonedds/__library__.py", line 21, in <module>
    library_path = list((dir / ".." / "cyclonedds.libs").glob("libddsc*"))[0]
                   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range

Looking closer at the built .whl, there is no cyclonedds.libs directory, but there is one named .libs. Am I perhaps using the wrong tools here? Inside the packaged .libs directory there is an idlc executable, but just not as self-contained as I had hoped, the dependencies don't match what is packaged:

tonic@funbox:~/src/cyclonedds-python/ex$ ls -l cyclonedds/.libs/
total 7844
-rwxr-xr-x 1 tonic tonic  515800 Feb  4 19:48 idlc
-rw-r--r-- 1 tonic tonic 7513896 Feb  4 19:48 libddsc.so

tonic@funbox:~/src/cyclonedds-python/ex$ ldd cyclonedds/.libs/idlc 
        linux-vdso.so.1 (0x00007ffe62b71000)
        libgtk3-nocsd.so.0 => /usr/lib/x86_64-linux-gnu/libgtk3-nocsd.so.0 (0x00007f8cac1e9000)
        libcycloneddsidl.so.0 => /home/tonic/local/lib/libcycloneddsidl.so.0 (0x00007f8cac1b8000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f8cabfd7000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f8cabfd2000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f8cabfcb000)
        libddsc.so.0 => /home/tonic/local/lib/libddsc.so.0 (0x00007f8cabe7b000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f8cac29b000)
        libssl.so.3 => /usr/lib/x86_64-linux-gnu/libssl.so.3 (0x00007f8cabdd2000)
        libcrypto.so.3 => /usr/lib/x86_64-linux-gnu/libcrypto.so.3 (0x00007f8cab800000)

Shouldn't libcycloneddsidl.so.0 also be packaged with the wheel along idlc? Also, the packaged libddsc.so seems to have wrong name, the linker looks for a libddsc.so.0.

Tested under Linux, x86_64, python 3.11.2 with a somewhat recent CycloneDDS master.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the STANDALONE_WHEELS=1 pip wheel --no-deps . build and the wheel_idlc command, then inspect cyclonedds/library.py and the packaging logic that creates cyclonedds/.libs. Done means the standalone wheel contains the native libraries under the paths and names expected at runtime, and the reported command works without relying on external local libraries.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.