openwrt / openwrt/packages

Getting issue while building custom python package 23.05

Open
#26,418 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

release/23.05
Dominant language
Makefile
Stars
4.6k
Forks
4k
Avg merge
3d 12h
Merged PRs (30d)
134

Description

I have recently upgraded from openwrt21.02 to openwrt23.05 (upgrade planned to 24.x)
I have a custom git rep python package which has setup.py and set of files which needs to be installed. they are present in feeds package which installs git repo and set of python files

Following was working with 3.9 however with openwrt 23.05, 3.11 python this complains about .whl unable to find it.
I know that with 3.11 stricter pep enforcements have been enforced. How can this be solved?

define Py3Package/runtests/install
# bin
$(INSTALL_DIR) $(1)/usr/bin
$(INSTALL_BIN) $(PKG_INSTALL_DIR)/usr/bin/runtests $(1)/usr/bin

$(INSTALL_DIR) $(1)/usr/lib/runtests
$(CP) $(PKG_INSTALL_DIR)/usr/lib/runtests $(1)/usr/lib

# Tests
$(INSTALL_DIR) $(1)$(PYTHON3_PKG_DIR)/tests
$(CP) files/usr/lib/python/tests/* $(1)$(PYTHON3_PKG_DIR)/tests

endef


* Building wheel...
Warning: 'classifiers' should be a list, got type 'tuple'
/home/mfw/openwrt/staging_dir/hostpkg/lib/python3.11/site-packages/setuptools/dist.py:544: UserWarning: The version specified ('undefined') is an invalid version, this may not work as expected with newer versions of setuptools, pip, and PyPI. Please see PEP 440 for more details.
  warnings.warn(
running bdist_wheel
/home/mfw/openwrt/staging_dir/hostpkg/lib/python3.11/site-packages/pkg_resources/__init__.py:123: PkgResourcesDeprecationWarning: undefined is an invalid version and will not be supported in a future release
  warnings.warn(
running build
running build_py
creating build
creating build/lib
creating build/lib/runtests
copying runtests/__init__.py -> build/lib/runtests
copying runtests/test_registry.py -> build/lib/runtests
copying runtests/remote_control.py -> build/lib/runtests
copying runtests/overrides.py -> build/lib/runtests
copying runtests/version.py -> build/lib/runtests
warning: build_py: byte-compiling is disabled, skipping.

running build_scripts
creating build/scripts-3.11
copying and adjusting bin/runtests -> build/scripts-3.11
changing mode of build/scripts-3.11/runtests from 644 to 755
installing to build/bdist.linux-x86_64/wheel
running install
running install_lib
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/wheel
creating build/bdist.linux-x86_64/wheel/runtests
copying build/lib/runtests/__init__.py -> build/bdist.linux-x86_64/wheel/runtests
copying build/lib/runtests/test_registry.py -> build/bdist.linux-x86_64/wheel/runtests
copying build/lib/runtests/remote_control.py -> build/bdist.linux-x86_64/wheel/runtests
copying build/lib/runtests/overrides.py -> build/bdist.linux-x86_64/wheel/runtests
copying build/lib/runtests/version.py -> build/bdist.linux-x86_64/wheel/runtests
warning: install_lib: byte-compiling is disabled, skipping.

running install_data
creating build/bdist.linux-x86_64/wheel/runtests-undefined.data
creating build/bdist.linux-x86_64/wheel/runtests-undefined.data/data
creating build/bdist.linux-x86_64/wheel/usr
creating build/bdist.linux-x86_64/wheel/usr/lib
creating build/bdist.linux-x86_64/wheel/usr/lib/runtests
copying files/test_shell.key -> build/bdist.linux-x86_64/wheel/usr/lib/runtests
copying files/test_shell.pub -> build/bdist.linux-x86_64/wheel/usr/lib/runtests
running install_egg_info
running egg_info
writing runtests.egg-info/PKG-INFO
writing dependency_links to runtests.egg-info/dependency_links.txt
writing top-level names to runtests.egg-info/top_level.txt
reading manifest file 'runtests.egg-info/SOURCES.txt'
adding license file 'LICENSE'
writing manifest file 'runtests.egg-info/SOURCES.txt'
Copying runtests.egg-info to build/bdist.linux-x86_64/wheel/runtests-undefined-py3.11.egg-info
running install_scripts
creating build/bdist.linux-x86_64/wheel/runtests-undefined.data/scripts
copying build/scripts-3.11/runtests -> build/bdist.linux-x86_64/wheel/runtests-undefined.data/scripts
changing mode of build/bdist.linux-x86_64/wheel/runtests-undefined.data/scripts/runtests to 755
creating build/bdist.linux-x86_64/wheel/runtests-undefined.dist-info/WHEEL
creating '/home/mfw/openwrt/build_dir/target-x86_64_musl/runtests-c8451db4b1fd97784d204b4d6746c43b7a3a323a/openwrt-build/tmp76ns7q1l/runtests-undefined-py3-none-any.whl' and adding 'build/bdist.linux-x86_64/wheel' to it
adding 'runtests/__init__.py'
adding 'runtests/overrides.py'
adding 'runtests/remote_control.py'
adding 'runtests/test_registry.py'
adding 'runtests/version.py'
adding 'runtests-undefined.data/scripts/runtests'
adding 'usr/lib/runtests/test_shell.key'
adding 'usr/lib/runtests/test_shell.pub'
adding 'runtests-undefined.dist-info/LICENSE'
adding 'runtests-undefined.dist-info/METADATA'
adding 'runtests-undefined.dist-info/WHEEL'
adding 'runtests-undefined.dist-info/top_level.txt'
adding 'runtests-undefined.dist-info/RECORD'
removing build/bdist.linux-x86_64/wheel
Successfully built runtests-undefined-py3-none-any.whl
cd "/home/mfw/openwrt/build_dir/target-x86_64_musl/runtests-c8451db4b1fd97784d204b4d6746c43b7a3a323a" && CC="x86_64-openwrt-linux-musl-gcc" CCSHARED="x86_64-openwrt-linux-musl-gcc -DPIC -fpic" CXX="x86_64-openwrt-linux-musl-g++" LD="x86_64-openwrt-linux-musl-gcc" LDSHARED="x86_64-openwrt-linux-musl-gcc -shared" CFLAGS="-Os -pipe -fno-caller-saves -fno-plt -fhonour-copts -fmacro-prefix-map=/home/mfw/openwrt/build_dir/target-x86_64_musl/runtests-c8451db4b1fd97784d204b4d6746c43b7a3a323a=runtests-c8451db4b1fd97784d204b4d6746c43b7a3a323a -Wformat -Werror=format-security -fstack-protector -D_FORTIFY_SOURCE=1 -Wl,-z,now -Wl,-z,relro" CPPFLAGS="-I/home/mfw/openwrt/staging_dir/toolchain-x86_64_gcc-12.3.0_musl/usr/include -I/home/mfw/openwrt/staging_dir/toolchain-x86_64_gcc-12.3.0_musl/include/fortify -I/home/mfw/openwrt/staging_dir/toolchain-x86_64_gcc-12.3.0_musl/include -I/home/mfw/openwrt/staging_dir/target-x86_64_musl/usr/include/python3.11" LDFLAGS="-L/home/mfw/openwrt/staging_dir/toolchain-x86_64_gcc-12.3.0_musl/usr/lib -L/home/mfw/openwrt/staging_dir/toolchain-x86_64_gcc-12.3.0_musl/lib -znow -zrelro -lpython3.11" _PYTHON_HOST_PLATFORM="linux-x86_64" _PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata__linux_x86_64-linux-musl" PYTHONPATH="/home/mfw/openwrt/staging_dir/target-x86_64_musl/usr/lib/python3.11:/home/mfw/openwrt/staging_dir/target-x86_64_musl//usr/lib/python3.11/site-packages:/home/mfw/openwrt/build_dir/target-x86_64_musl/runtests-c8451db4b1fd97784d204b4d6746c43b7a3a323a/ipkg-install//usr/lib/python3.11/site-packages" PYTHONDONTWRITEBYTECODE=1 _python_sysroot="/home/mfw/openwrt/staging_dir/target-x86_64_musl" _python_prefix="/usr" _python_exec_prefix="/usr"  CARGO_BUILD_TARGET=x86_64-unknown-linux-musl CARGO_HOME=/home/mfw/openwrt/dl/cargo CARGO_PROFILE_RELEASE_CODEGEN_UNITS=1 CARGO_PROFILE_RELEASE_DEBUG=false CARGO_PROFILE_RELEASE_DEBUG_ASSERTIONS=false CARGO_PROFILE_RELEASE_LTO=true CARGO_PROFILE_RELEASE_OPT_LEVEL=z CARGO_PROFILE_RELEASE_OVERFLOW_CHECKS=true CARGO_PROFILE_RELEASE_PANIC=unwind CARGO_PROFILE_RELEASE_RPATH=false CARGO_TARGET_X86_64_UNKNOWN_LINUX_MUSL_LINKER=x86_64-openwrt-linux-musl-gcc RUSTFLAGS="-Ctarget-feature=-crt-static -lssp_nonshared" TARGET_CC=x86_64-openwrt-linux-musl-gcc TARGET_CFLAGS="-Os -pipe -fno-caller-saves -fno-plt -fhonour-copts -fmacro-prefix-map=/home/mfw/openwrt/build_dir/target-x86_64_musl/runtests-c8451db4b1fd97784d204b4d6746c43b7a3a323a=runtests-c8451db4b1fd97784d204b4d6746c43b7a3a323a -Wformat -Werror=format-security -fstack-protector -D_FORTIFY_SOURCE=1 -Wl,-z,now -Wl,-z,relro " PYO3_CROSS_LIB_DIR="/home/mfw/openwrt/staging_dir/target-x86_64_musl/usr/lib/python3.11" SETUPTOOLS_RUST_CARGO_PROFILE="release"    /home/mfw/openwrt/staging_dir/hostpkg/bin/python3.11  -m installer --destdir "/home/mfw/openwrt/build_dir/target-x86_64_musl/runtests-c8451db4b1fd97784d204b4d6746c43b7a3a323a/ipkg-install" --no-compile-bytecode --prefix /usr "/home/mfw/openwrt/build_dir/target-x86_64_musl/runtests-c8451db4b1fd97784d204b4d6746c43b7a3a323a/"/openwrt-build/runtests-c8451db4b1fd97784d204b4d6746c43b7a3a323a-*.whl 
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/mfw/openwrt/staging_dir/hostpkg/lib/python3.11/site-packages/installer/__main__.py", line 98, in <module>
    _main(sys.argv[1:], "python -m installer")
  File "/home/mfw/openwrt/staging_dir/hostpkg/lib/python3.11/site-packages/installer/__main__.py", line 86, in _main
    with WheelFile.open(args.wheel) as source:
  File "/home/mfw/openwrt/staging_dir/target-x86_64_musl/usr/lib/python3.11/contextlib.py", line 137, in __enter__
    return next(self.gen)
           ^^^^^^^^^^^^^^
  File "/home/mfw/openwrt/staging_dir/hostpkg/lib/python3.11/site-packages/installer/sources.py", line 162, in open
    with zipfile.ZipFile(path) as f:
         ^^^^^^^^^^^^^^^^^^^^^
  File "/home/mfw/openwrt/staging_dir/target-x86_64_musl/usr/lib/python3.11/zipfile.py", line 1284, in __init__
    self.fp = io.open(file, filemode)
              ^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/home/mfw/openwrt/build_dir/target-x86_64_musl/runtests-c8451db4b1fd97784d204b4d6746c43b7a3a323a//openwrt-build/runtests-c8451db4b1fd97784d204b4d6746c43b7a3a323a-*.whl'
make[3]: *** [Makefile:47: /home/mfw/openwrt/build_dir/target-x86_64_musl/runtests-c8451db4b1fd97784d204b4d6746c43b7a3a323a/.built] Error 1
make[3]: Leaving directory '/home/mfw/openwrt/feeds/mfw/runtests'
time: package/feeds/mfw/runtests/compile#3.28#0.62#16.35
    ERROR: package/feeds/mfw/runtests failed to build.
make[2]: *** [package/Makefile:129: package/feeds/mfw/runtests/compile] Error 1
make[2]: Leaving directory '/home/mfw/openwrt'
make[1]: *** [package/Makefile:123: /home/mfw/openwrt/staging_dir/target-x86_64_musl/stamp/.package_compile] Error 2
make[1]: Leaving directory '/home/mfw/openwrt'

make: *** [/home/mfw/openwrt/include/toplevel.mk:233: world] Error 2

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with feeds/mfw/runtests/Makefile at the failing line 47 and compare its wheel path with the wheel filename produced by setup.py. Run the runtests package build on OpenWrt 23.05 and inspect the generated wheel and installer command. Done means the package build locates the generated wheel and completes installation.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.