kivy / kivy/python-for-android
`charset-normalizer >= 3.0.0` is not plain-python anymore, and requires a recipe
- Dominant language
- Python
- Stars
- 8.9k
- Forks
- 2k
- Avg merge
- 12h 20m
- Merged PRs (30d)
- 9
Description
### Checklist
- [x] the issue is indeed a bug and not a support request
- [x] issue doesn't already exist: https://github.com/kivy/python-for-android/issues
- [x] I have a short, runnable example that reproduces the issue
- [ ] I reproduced the problem with the latest development version (`p4a.branch = develop`)
- [x] I used the grave accent (aka backticks) to format code or logs when appropriated
### Versions
- Python: 3.8.10
- OS: Ubuntu 20.0.4 on WSL2 ( WIndows11 )
- Kivy: 2.10
- Cython: 0.29.33
- OpenJDK: 17.0.5
### Description
The app should display repeatedly some stock quotes using Kivy and yahooquery module. It works on windows.
Buildozer build process produces an apk. When launched on the smartphone, the app window doesn't show up. adb logs report the following error:
``` 02-21 18:02:40.903 12625 12674 I python : File "/home/ced/programmation/python/kivy/kivy_bourse/.buildozer/android/platform/build-arm64-v8a_armeabi-v7a/build/python-installs/Bourse/arm64-v8a/charset_normalizer/cd.py", line 9, in
02-21 18:02:40.903 12625 12674 I python : ImportError: dlopen failed: "/data/data/ced.bart.bourse/files/app/_python_bundle/site-packages/charset_normalizer/md.so" is for EM_X86_64 (62) instead of EM_AARCH64 (183)
02-21 18:02:40.903 12625 12674 I python : Python for android ended.
```
### buildozer.spec
Command:
```sh
buildozer -v android debug
```
Spec file:
```
[app]
title = Bourse
package.name = Bourse
package.domain = ced.bart
source.dir = .
source.include_exts = py,png,jpg,kv,atlas,json,wav
version = 0.1
requirements=python3,kivy==2.1.0,datetime,yahooquery,pandas,lxml,numpy,python-dateutil,six,pytz,requests-futures,requests,certifi,charset-normalizer,idna,urllib3,tqdm
orientation = portrait
osx.python_version = 3
osx.kivy_version = 2.1.0
fullscreen = 0
android.archs = arm64-v8a, armeabi-v7a
android.allow_backup = True
ios.kivy_ios_url = https://github.com/kivy/kivy-ios
ios.kivy_ios_branch = master
ios.ios_deploy_url = https://github.com/phonegap/ios-deploy
ios.ios_deploy_branch = 1.10.0
ios.codesign.allowed = false
[buildozer]
log_level = 2
warn_on_root = 1
```
### Logs
```
Tue Feb 21 19:02:14 CET 2023
# Check configuration tokens
# Ensure build layout
# Check configuration tokens
# Preparing build
# Check requirements for android
# Search for Git (git)
# -> found at /usr/bin/git
# Search for Cython (cython)
# -> found at /home/ced/.local/bin/cython
# Search for Java compiler (javac)
# -> found at /usr/lib/jvm/java-17-openjdk-amd64/bin/javac
# Search for Java keytool (keytool)
# -> found at /usr/lib/jvm/java-17-openjdk-amd64/bin/keytool
# Install platform
# Run ['git', 'config', '--get', 'remote.origin.url']
# Cwd /home/ced/programmation/python/kivy/kivy_bourse/.buildozer/android/platform/python-for-android
https://github.com/kivy/python-for-android.git
# Run ['git', 'branch', '-vv']
# Cwd /home/ced/programmation/python/kivy/kivy_bourse/.buildozer/android/platform/python-for-android
* master 56bf7335 [origin/master] Merge pull request #2747 from misl6/release-2023.02.10
# Run ['/usr/bin/python', '-m', 'pip', 'install', '-q', '--user', 'appdirs', 'colorama>=0.3.3', 'jinja2', 'sh>=1.10, <2.0; sys_platform!="nt"', 'pep517', 'toml', 'packaging']
# Cwd None
WARNING: Ignoring invalid distribution -ip (/home/ced/.local/lib/python3.8/site-packages)
WARNING: Ignoring invalid distribution -ip (/home/ced/.local/lib/python3.8/site-packages)
WARNING: Ignoring invalid distribution -ip (/home/ced/.local/lib/python3.8/site-packages)
WARNING: Ignoring invalid distribution -ip (/home/ced/.local/lib/python3.8/site-packages)
WARNING: Ignoring invalid distribution -ip (/home/ced/.local/lib/python3.8/site-packages)
WARNING: Ignoring invalid distribution -ip (/home/ced/.local/lib/python3.8/site-packages)
[notice] A new release of pip available: 22.3.1 -> 23.0
[notice] To update, run: python -m pip install --upgrade pip
# Apache ANT found at /home/ced/.buildozer/android/platform/apache-ant-1.9.4
# Android SDK found at /home/ced/.buildozer/android/platform/android-sdk
# Recommended android's NDK version by p4a is: 25b
# Android NDK found at /home/ced/.buildozer/android/platform/android-ndk-r25b
# Run ['/usr/bin/python', '-m', 'pythonforandroid.toolchain', 'aab', '-h', '--color=always', '--storage-dir=/home/ced/programmation/python/kivy/kivy_bourse/.buildozer/android/platform/build-arm64-v8a_armeabi-v7a', '--ndk-api=21', '--ignore-setup-py', '--debug']
# Cwd /home/ced/programmation/python/kivy/kivy_bourse/.buildozer/android/platform/python-for-android
[WARNING]: prerequisites.py is experimental and does not support all prerequisites yet.
[WARNING]: Please report any issues to the python-for-android issue tracker.
/home/ced/programmation/python/kivy/kivy_bourse/.buildozer/android/platform/python-for-android/pythonforandroid/toolchain.py:59: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
if LooseVersion(cur_ver) < LooseVersion(version):
[WARNING]: prerequisites.py is experimental and does not support all prerequisites yet.
[WARNING]: Please report any issues to the python-for-android issue tracker.
usage: toolchain.py aab [-h] [--debug] [--color {always,never,auto}]
[--sdk-dir SDK_DIR] [--ndk-dir NDK_DIR]
[--android-api ANDROID_API]
[--ndk-version NDK_VERSION] [--ndk-api NDK_API]
[--symlink-bootstrap-files]
[--storage-dir STORAGE_DIR] [--arch ARCH]
[--dist-name DIST_NAME] [--requirements REQUIREMENTS]
[--recipe-blacklist RECIPE_BLACKLIST]
[--blacklist-requirements BLACKLIST_REQUIREMENTS]
[--bootstrap BOOTSTRAP] [--hook HOOK] [--force-build]
[--no-force-build] [--require-perfect-match]
[--no-require-perfect-match] [--allow-replace-dist]
[--no-allow-replace-dist]
[--local-recipes LOCAL_RECIPES]
[--activity-class-name ACTIVITY_CLASS_NAME]
[--service-class-name SERVICE_CLASS_NAME]
[--java-build-tool {auto,ant,gradle}] [--copy-libs]
[--no-copy-libs] [--add-asset ASSETS]
[--add-resource RESOURCES] [--private PRIVATE]
[--use-setup-py] [--ignore-setup-py] [--release]
[--with-debug-symbols] [--keystore KEYSTORE]
[--signkey SIGNKEY] [--keystorepw KEYSTOREPW]
[--signkeypw SIGNKEYPW]
optional arguments:
-h, --help show this help message and exit
--debug Display debug output and all build info
--color {always,never,auto}
Enable or disable color output (default enabled on
tty)
--sdk-dir SDK_DIR, --sdk_dir SDK_DIR
The filepath where the Android SDK is installed
--ndk-dir NDK_DIR, --ndk_dir NDK_DIR
The filepath where the Android NDK is installed
--android-api ANDROID_API, --android_api ANDROID_API
The Android API level to build against defaults to 33
if not specified.
--ndk-version NDK_VERSION, --ndk_version NDK_VERSION
DEPRECATED: the NDK version is now found automatically
or not at all.
--ndk-api NDK_API The Android API level to compile against. This should
be your *minimal supported* API, not normally the same
as your --android-api. Defaults to min(ANDROID_API,
21) if not specified.
--symlink-bootstrap-files, --ssymlink_bootstrap_files
If True, symlinks the bootstrap files creation. This
is useful for development only, it could also cause
weird problems.
--storage-dir STORAGE_DIR
Primary storage directory for downloads and builds
(default: /home/ced/.local/share/python-for-android)
--arch ARCH The archs to build for.
--dist-name DIST_NAME, --dist_name DIST_NAME
The name of the distribution to use or create
--requirements REQUIREMENTS
Dependencies of your app, should be recipe names or
Python modules. NOT NECESSARY if you are using Python
3 with --use-setup-py
--recipe-blacklist RECIPE_BLACKLIST
Blacklist an internal recipe from use. Allows
disabling Python 3 core modules to save size
--blacklist-requirements BLACKLIST_REQUIREMENTS
Blacklist an internal recipe from use. Allows
disabling Python 3 core modules to save size
--bootstrap BOOTSTRAP
The bootstrap to build with. Leave unset to choose
automatically.
--hook HOOK Filename to a module that contains python-for-android
hooks
--local-recipes LOCAL_RECIPES, --local_recipes LOCAL_RECIPES
Directory to look for local recipes
--activity-class-name ACTIVITY_CLASS_NAME
The full java class name of the main activity
--service-class-name SERVICE_CLASS_NAME
Full java package name of the PythonService class
--java-build-tool {auto,ant,gradle}
The java build tool to use when packaging the APK,
defaults to automatically selecting an appropriate
tool.
--add-asset ASSETS Put this in the assets folder in the apk.
--add-resource RESOURCES
Put this in the res folder in the apk.
--private PRIVATE the directory with the app source code files
(containing your main.py entrypoint)
--use-setup-py Process the setup.py of a project if present.
(Experimental!
--ignore-setup-py Don't run the setup.py of a project if present. This
may be required if the setup.py is not designed to
work inside p4a (e.g. by installing dependencies that
won't work or aren't desired on Android
--release Build your app as a non-debug release build. (Disables
gdb debugging among other things)
--with-debug-symbols Will keep debug symbols from `.so` files.
--keystore KEYSTORE Keystore for JAR signing key, will use jarsigner
default if not specified (release build only)
--signkey SIGNKEY Key alias to sign PARSER_APK. with (release build
only)
--keystorepw KEYSTOREPW
Password for keystore
--signkeypw SIGNKEYPW
Password for key alias
Whether to force compilation of a new distribution
--force-build
--no-force-build (this is the default)
--require-perfect-match
--no-require-perfect-match
(this is the default)
--allow-replace-dist (this is the default)
--no-allow-replace-dist
--copy-libs
--no-copy-libs (this is the default)
# Check application requirements
# Compile platform
# Run ['/usr/bin/python', '-m', 'pythonforandroid.toolchain', 'create', '--dist_name=Bourse', '--bootstrap=sdl2', '--requirements=python3,kivy==2.1.0,datetime,yahooquery,pandas,lxml,numpy,python-dateutil,six,pytz,requests-futures,requests,certifi,charset-normalizer==2.1.1,idna,urllib3,tqdm', '--arch=arm64-v8a', '--arch=armeabi-v7a', '--copy-libs', '--color=always', '--storage-dir=/home/ced/programmation/python/kivy/kivy_bourse/.buildozer/android/platform/build-arm64-v8a_armeabi-v7a', '--ndk-api=21', '--ignore-setup-py', '--debug']
# Cwd /home/ced/programmation/python/kivy/kivy_bourse/.buildozer/android/platform/python-for-android
[WARNING]: prerequisites.py is experimental and does not support all prerequisites yet.
[WARNING]: Please report any issues to the python-for-android issue tracker.
/home/ced/programmation/python/kivy/kivy_bourse/.buildozer/android/platform/python-for-android/pythonforandroid/toolchain.py:59: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
if LooseVersion(cur_ver) < LooseVersion(version):
[WARNING]: prerequisites.py is experimental and does not support all prerequisites yet.
[WARNING]: Please report any issues to the python-for-android issue tracker.
[INFO]: Recipe kivy: version "2.1.0" requested
[INFO]: Recipe charset-normalizer: version "2.1.1" requested
[WARNING]: $ANDROIDNDKVER is deprecated and no longer necessary, the value you set is ignored
[INFO]: Will compile for the following archs: armeabi-v7a, arm64-v8a
[INFO]: Found Android API target in $ANDROIDAPI: 31
[INFO]: Available Android APIs are (31)
[INFO]: Requested API target 31 is available, continuing.
[INFO]: Found NDK dir in $ANDROIDNDK: /home/ced/.buildozer/android/platform/android-ndk-r25b
[INFO]: Found NDK version 25b
[INFO]: Getting NDK API version (i.e. minimum supported API) from user argument
[INFO]: Of the existing distributions, the following meet the given requirements:
[INFO]: Bourse: min API 21, includes recipes (hostpython3, libbz2, libffi, liblzma, librt, libxml2, openssl, sdl2_image, sdl2_mixer, sdl2_ttf, sqlite3, libxslt, python3, sdl2, pytz, setuptools, cython, lxml, six, numpy, pyjnius, android, kivy, pandas, tqdm, requests, urllib3, datetime, charset-normalizer, certifi, requests-futures, yahooquery, python-dateutil, idna), built for archs (armeabi-v7a, arm64-v8a)
[INFO]: Bourse has compatible recipes, using this one
# Build the application #8
# Copy application source from /home/ced/programmation/python/kivy/kivy_bourse
# Create directory /home/ced/programmation/python/kivy/kivy_bourse/.buildozer/android/app
# Copy /home/ced/programmation/python/kivy/kivy_bourse/main.py
# Copy /home/ced/programmation/python/kivy/kivy_bourse/recurse.py
# Package the application
# project.properties updated
# Run ['/usr/bin/python', '-m', 'pythonforandroid.toolchain', 'apk', '--bootstrap', 'sdl2', '--dist_name', 'Bourse', '--name', 'Bourse', '--version', '0.1', '--package', 'ced.bart.bourse', '--minsdk', '21', '--ndk-api', '21', '--private', '/home/ced/programmation/python/kivy/kivy_bourse/.buildozer/android/app', '--android-entrypoint', 'org.kivy.android.PythonActivity', '--android-apptheme', '@android:style/Theme.NoTitleBar', '--orientation', 'portrait', '--window', '--enable-androidx', '--copy-libs', '--arch', 'arm64-v8a', '--arch', 'armeabi-v7a', '--color=always', '--storage-dir=/home/ced/programmation/python/kivy/kivy_bourse/.buildozer/android/platform/build-arm64-v8a_armeabi-v7a', '--ndk-api=21', '--ignore-setup-py', '--debug']
# Cwd /home/ced/programmation/python/kivy/kivy_bourse/.buildozer/android/platform/python-for-android
[WARNING]: prerequisites.py is experimental and does not support all prerequisites yet.
[WARNING]: Please report any issues to the python-for-android issue tracker.
/home/ced/programmation/python/kivy/kivy_bourse/.buildozer/android/platform/python-for-android/pythonforandroid/toolchain.py:59: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
if LooseVersion(cur_ver) < LooseVersion(version):
[WARNING]: prerequisites.py is experimental and does not support all prerequisites yet.
[WARNING]: Please report any issues to the python-for-android issue tracker.
[WARNING]: $ANDROIDNDKVER is deprecated and no longer necessary, the value you set is ignored
[INFO]: Will compile for the following archs: armeabi-v7a, arm64-v8a
[INFO]: Found Android API target in $ANDROIDAPI: 31
[INFO]: Available Android APIs are (31)
[INFO]: Requested API target 31 is available, continuing.
[INFO]: Found NDK dir in $ANDROIDNDK: /home/ced/.buildozer/android/platform/android-ndk-r25b
[INFO]: Found NDK version 25b
[INFO]: Getting NDK API version (i.e. minimum supported API) from user argument
[INFO]: Of the existing distributions, the following meet the given requirements:
[INFO]: Bourse: min API 21, includes recipes (hostpython3, libbz2, libffi, liblzma, librt, libxml2, openssl, sdl2_image, sdl2_mixer, sdl2_ttf, sqlite3, libxslt, python3, sdl2, pytz, setuptools, cython, lxml, six, numpy, pyjnius, android, kivy, pandas, tqdm, requests, urllib3, datetime, charset-normalizer, certifi, requests-futures, yahooquery, python-dateutil, idna), built for archs (armeabi-v7a, arm64-v8a)
[INFO]: Bourse has compatible recipes, using this one
[INFO]: Of the existing distributions, the following meet the given requirements:
[INFO]: Bourse: min API 21, includes recipes (hostpython3, libbz2, libffi, liblzma, librt, libxml2, openssl, sdl2_image, sdl2_mixer, sdl2_ttf, sqlite3, libxslt, python3, sdl2, pytz, setuptools, cython, lxml, six, numpy, pyjnius, android, kivy, pandas, tqdm, requests, urllib3, datetime, charset-normalizer, certifi, requests-futures, yahooquery, python-dateutil, idna), built for archs (armeabi-v7a, arm64-v8a)
[INFO]: Bourse has compatible recipes, using this one
[INFO]: -> directory context /home/ced/programmation/python/kivy/kivy_bourse/.buildozer/android/platform/build-arm64-v8a_armeabi-v7a/build/bootstrap_builds/sdl2
[INFO]: <- directory context /home/ced/programmation/python/kivy/kivy_bourse/.buildozer/android/platform/python-for-android
[INFO]: -> directory context /home/ced/programmation/python/kivy/kivy_bourse/.buildozer/android/platform/build-arm64-v8a_armeabi-v7a/dists/Bourse
Compiling '/home/ced/programmation/python/kivy/kivy_bourse/.buildozer/android/app/main.py'...
Compiling '/home/ced/programmation/python/kivy/kivy_bourse/.buildozer/android/app/sitecustomize.py'...
Compiling '/home/ced/programmation/python/kivy/kivy_bourse/.buildozer/android/app/recurse.py'...
[INFO]: Detected highest available build tools version to be 34.0.0-rc1
[DEBUG]: -> running dos2unix /home/ced/programmation/python/kivy/kivy_bourse/.buildozer/android/platform/build-arm64-v8a_armeabi-v7a/dists/Bourse/gradlew
[DEBUG]: dos2unix: converting file /home/ced/programmation/python/kivy/kivy_bourse/.buildozer/android/platform/build-arm64-v8a_armeabi-v7a/dists/Bourse/gradlew to Unix format...
[DEBUG]: -> running gradlew clean assembleDebug
[DEBUG]:
[DEBUG]: > Configure project :
[DEBUG]: WARNING:Using flatDir should be avoided because it doesn't support any meta-data formats.
[DEBUG]:
[DEBUG]: > Task :compileDebugJavaWithJavac
[DEBUG]: Note: Some input files use or override a deprecated API.
[DEBUG]: Note: Recompile with -Xlint:deprecation for details.
[DEBUG]:
[DEBUG]: > Task :stripDebugDebugSymbols
[DEBUG]: Unable to strip the following libraries, packaging them as they are: libSDL2.so, libSDL2_image.so, libSDL2_mixer.so, libSDL2_ttf.so, libbz2.so, libc++_shared.so, libcrypto1.1.so, libffi.so, liblzma.so, libmain.so, libpybundle.so, libpython3.9.so, libsqlite3.so, libssl1.1.so.
[DEBUG]:
[DEBUG]: Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
[DEBUG]:
[DEBUG]: You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.DLE
[DEBUG]:
[DEBUG]: See https://docs.gradle.org/7.4.1/userguide/command_line_interface.html#sec:command_line_warnings
[DEBUG]:
[DEBUG]: BUILD SUCCESSFUL in 18s
[DEBUG]: 31 actionable tasks: 31 executed
[INFO]: <- directory context /home/ced/programmation/python/kivy/kivy_bourse/.buildozer/android/platform/python-for-android
[INFO]: Of the existing distributions, the following meet the given requirements:
[INFO]: Bourse: min API 21, includes recipes (hostpython3, libbz2, libffi, liblzma, librt, libxml2, openssl, sdl2_image, sdl2_mixer, sdl2_ttf, sqlite3, libxslt, python3, sdl2, pytz, setuptools, cython, lxml, six, numpy, pyjnius, android, kivy, pandas, tqdm, requests, urllib3, datetime, charset-normalizer, certifi, requests-futures, yahooquery, python-dateutil, idna), built for archs (armeabi-v7a, arm64-v8a)
[INFO]: Bourse has compatible recipes, using this one
[INFO]: # Copying android package to current directory
[INFO]: # Android package filename not found in build output. Guessing...
[INFO]: # Found android package file: /home/ced/programmation/python/kivy/kivy_bourse/.buildozer/android/platform/build-arm64-v8a_armeabi-v7a/dists/Bourse/build/outputs/apk/debug/Bourse-debug.apk
[INFO]: # Add version number to android package
[INFO]: # Android package renamed to Bourse-debug-0.1.apk
[DEBUG]: -> running cp /home/ced/programmation/python/kivy/kivy_bourse/.buildozer/android/platform/build-arm64-v8a_armeabi-v7a/dists/Bourse/build/outputs/apk/debug/Bourse-debug.apk Bourse-debug-0.1.apk
WARNING: Received a --sdk argument, but this argument is deprecated and does nothing.
No setup.py/pyproject.toml used, copying full private data into .apk.
Applying Java source code patches...
Applying patch: src/patches/SDLActivity.java.patch
Warning: failed to apply patch (exit code 1), assuming it is already applied: src/patches/SDLActivity.java.patch
# Android packaging done!
# APK Bourse-0.1-arm64-v8a_armeabi-v7a-debug.apk available in the bin directory
```
Contributor guide
Research direction
Start by reproducing the reported `buildozer -v android debug` build with `charset-normalizer` in the requirements, then inspect `pythonforandroid/toolchain.py` and the recipe-related options shown in its command output. Done means the Android APK builds and launches without the `md.so` architecture mismatch reported in the logs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- android, python
- Domain
- build-system, mobile
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100