Multi platform pex distribution not supported
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 603
- Forks
- 142
- PR merge metrics
- No merged PRs in 30d
Description
Eventhough we can set multiple platforms via pexOptions for pex builder, the wheels dependencies are not downloaded. This off course ends with an exception:
`11:24:40.539 [INFO] [org.gradle.process.internal.DefaultExecHandle] Starting process 'command '/workspace/tr/build/venv/bin/python''. Working directory: /workspace/tr Command: /workspace/tr/build/venv/bin/python /workspace/tr/build/venv/bin/pex --no-pypi --cache-dir /workspace/tr/build/pex-cache --output-file /workspace/tr/build/deployable/bin/test-runner --repo /build/wheel-cache --python-shebang /usr/bin/env python --entry-point tr.runner:main --platform=manylinux1-x86_64 --platform=macosx-10.6-x86_64 nose==1.3.7 pbr==3.1.1 pathtools==0.1.2 watchdog==0.8.3 PyYAML==3.12 lxml==4.1.1 pyserial==3.4 requests==2.11.1 Louie==1.1 test-runner==4.0.0 argh==0.26.2 psutil==5.4.3
11:24:40.539 [DEBUG] [org.gradle.process.internal.DefaultExecHandle] Environment for process 'command '/workspace/tr/build/venv/bin/python'': {PATH=/my.depot/main//Tools/3rdParty/MacOSX/itnl:/my.depot/main//Tools/3rdParty/MacOSX/ios-deploy:/my.depot/main//MacOSX/Build/cmake/3.8.1/bin:/my.depot/main//MacOSX/Build/ant/bin:/Library/Maven/apache-maven-3.3.9/bin:/Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/Applications/Wireshark.app/Contents/MacOS:/my.depot/main//MacOSX/Android/ndk9c:/my.depot/main//MacOSX/Android/sdk24/build-tools/23.0.1:/my.depot/main//MacOSX/Android/sdk24/platform-tools:/my.depot/main//MacOSX/Android/sdk24/tools, LC_MONETARY=en_US.UTF-8, LC_TIME=en_US.UTF-8, JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home, JAVA8_HOME=/Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home, LC_MESSAGES=en_US.UTF-8, TERM=xterm-256color, CT_DEVENV_HOME=/my.depot/main/, LANG=en_US.UTF-8, ANDROID_HOME=/my.depot/main//MacOSX/Android/sdk24, ANT_HOME=/my.depot/main//MacOSX/Build/ant, DISPLAY=/private/tmp/com.apple.launchd.wA0KSfYOF1/org.macosforge.xquartz:0, LC_COLLATE=en_US.UTF-8, LOGNAME=user, TERM_PROGRAM_VERSION=400, PWD=/workspace/tr, XPC_SERVICE_NAME=0, JAVA_MAIN_CLASS_31045=org.gradle.wrapper.GradleWrapperMain, NAVKIT_DATASET=/Volumes/Maps, SHELL=/bin/bash, TERM_PROGRAM=Apple_Terminal, JAVA7_HOME=/Library/Java/JavaVirtualMachines/jdk1.7.0_79.jdk/Contents/Home, LSCOLORS=ExFxBxDxCxegedabagacad, CT_DEVENV_SYSTEM_HOME=/my.depot/main//MacOSX, ANDROID_NDK_HOME=/my.depot/main//MacOSX/Android/ndk9c, APP_ICON_31045=/workspace/tr/media/gradle.icns, OLDPWD=/workspace/tr, SECURITYSESSIONID=186a8, CLICOLOR=1, USER=user, TMPDIR=/var/folders/6t/zrvxmx8101l60k1w6l1f9pcw0000gn/T/, LC_NUMERIC=en_US.UTF-8, SSH_AUTH_SOCK=/private/tmp/com.apple.launchd.pNNksY1bji/Listeners, NGL_DATASET=/Users/user/Perforce/user_mac_depot-ngl-dataset/, APP_NAME_31045=Gradle, XPC_FLAGS=0x0, TERM_SESSION_ID=FBE48B51-C6C9-44E2-8C26-0B4CDE74EED8, LC_ALL=, __CF_USER_TEXT_ENCODING=0x1F5:0x0:0x0, Apple_PubSub_Socket_Render=/private/tmp/com.apple.launchd.Xxicjt6IOD/Render, LC_CTYPE=en_US.UTF-8, HOME=/Users/user, SHLVL=1}
11:24:40.539 [DEBUG] [org.gradle.process.internal.DefaultExecHandle] Changing state to: STARTING
11:24:40.539 [DEBUG] [org.gradle.process.internal.DefaultExecHandle] Waiting until process started: command '/workspace/tr/build/venv/bin/python'.
11:24:40.563 [DEBUG] [org.gradle.process.internal.DefaultExecHandle] Changing state to: STARTED
11:24:40.563 [INFO] [org.gradle.process.internal.DefaultExecHandle] Successfully started process 'command '/workspace/tr/build/venv/bin/python''
11:24:40.563 [DEBUG] [org.gradle.process.internal.ExecHandleRunner] waiting until streams are handled...
11:24:41.373 [DEBUG] [org.gradle.process.internal.DefaultExecHandle] Changing state to: FAILED
11:24:41.374 [DEBUG] [org.gradle.process.internal.DefaultExecHandle] Process 'command '/workspace/tr/build/venv/bin/python'' finished with exit value 1 (state: FAILED)
11:24:41.374 [LIFECYCLE] [com.linkedin.gradle.python.util.internal.pex.PexExecOutputParser] Could not satisfy all requirements for watchdog==0.8.3:
watchdog==0.8.3
`
Additional setup for building pex:
`buildPex.pexOptions` = ["--platform=manylinux1-x86_64", "--platform=macosx-10.6-x86_64"]`
Wheel cache dir:
`$ ls build/wheel-cache/
Louie-1.1-py2-none-any.whl lxml-4.1.1-cp27-cp27m-macosx_10_13_intel.whl pbr-3.1.1-py2.py3-none-any.whl requests-2.11.1-py2-none-any.whl
PyYAML-3.12-cp27-cp27m-macosx_10_13_intel.whl nose-1.3.7-py2-none-any.whl psutil-5.4.3-cp27-cp27m-macosx_10_13_intel.whl test_runner-4.0.0-py2-none-any.whl
argh-0.26.2-py2.py3-none-any.whl pathtools-0.1.2-py2-none-any.whl pyserial-3.4-py2.py3-none-any.whl watchdog-0.8.3-cp27-cp27m-macosx_10_13_intel.whl`
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the shown pex command with both --platform values and inspect the pex builder path that processes pexOptions and the build/wheel-cache directory. Done means dependencies such as watchdog are downloaded or selected for each requested platform and the multi-platform PEX build completes successfully.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- build-system
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100