GoogleCloudPlatform / GoogleCloudPlatform/gsutil

Bundled python cannot use system pythons crcmod nowadays

Open
#1,429 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
918
Forks
335
PR merge metrics
No merged PRs in 30d

Description

```
[van@test-crcmod ~]$ sudo pip3 uninstall crcmod
Uninstalling crcmod-1.7:
/usr/local/lib64/python3.6/site-packages/crcmod-1.7-py3.6.egg-info
/usr/local/lib64/python3.6/site-packages/crcmod/__init__.py
/usr/local/lib64/python3.6/site-packages/crcmod/__pycache__/__init__.cpython-36.pyc
/usr/local/lib64/python3.6/site-packages/crcmod/__pycache__/_crcfunpy.cpython-36.pyc
/usr/local/lib64/python3.6/site-packages/crcmod/__pycache__/crcmod.cpython-36.pyc
/usr/local/lib64/python3.6/site-packages/crcmod/__pycache__/predefined.cpython-36.pyc
/usr/local/lib64/python3.6/site-packages/crcmod/__pycache__/test.cpython-36.pyc
/usr/local/lib64/python3.6/site-packages/crcmod/_crcfunext.cpython-36m-x86_64-linux-gnu.so
/usr/local/lib64/python3.6/site-packages/crcmod/_crcfunpy.py
/usr/local/lib64/python3.6/site-packages/crcmod/crcmod.py
/usr/local/lib64/python3.6/site-packages/crcmod/predefined.py
/usr/local/lib64/python3.6/site-packages/crcmod/test.py
Proceed (y/n)? y
Successfully uninstalled crcmod-1.7
[van@test-crcmod ~]$ sudo pip3 install --no-cache-dir -U crcmod
WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.
Collecting crcmod
Downloading https://files.pythonhosted.org/packages/6b/b0/e595ce2a2527e169c3bcd6c33d2473c1918e0b7f6826a043ca1245dd4e5b/crcmod-1.7.tar.gz (89kB)
100% |████████████████████████████████| 92kB 6.1MB/s
Installing collected packages: crcmod
Running setup.py install for crcmod ... done
/usr/local/lib/python3.6/site-packages/setuptools/command/install.py:37: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
setuptools.SetuptoolsDeprecationWarning,
Successfully installed crcmod-1.7
[van@test-crcmod ~]$ gsutil version -l
gsutil version: 5.5
checksum: 54a50a74649e27e7568328e5852187f0 (OK)
boto version: 2.49.0
python version: 3.8.11 (default, Sep 29 2021, 17:21:43) [Clang 12.0.1 ]
OS: Linux 3.10.0-1160.49.1.el7.x86_64
multiprocessing available: True
using cloud sdk: True
pass cloud sdk credentials to gsutil: True
config path(s): /etc/boto.cfg
gsutil path: /usr/lib64/google-cloud-sdk/bin/gsutil
compiled crcmod: False
installed via package manager: False
editable install: False
```

```
[van@test-crcmod ~]$ /usr/lib64/google-cloud-sdk/platform/bundledpythonunix/bin/python3
Python 3.8.11 (default, Sep 29 2021, 17:21:43)
[Clang 12.0.1 ] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import crcmod._crcfunext
Traceback (most recent call last):
File "", line 1, in
ModuleNotFoundError: No module named 'crcmod._crcfunext'
>>>
[van@test-crcmod ~]$ python3
Python 3.6.8 (default, Aug 13 2020, 07:46:32)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-39)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import crcmod._crcfunext
>>>
```

Same behavior I see on both RHEL7.9 and on Fedora 35

Contributor guide

Open the contributing guide

Research direction

Reproduce the import comparison using /usr/lib64/google-cloud-sdk/platform/bundledpythonunix/bin/python3 and the system python3, then inspect the crcmod behavior shown by gsutil version -l. Determine why the bundled interpreter cannot import crcmod._crcfunext while the system interpreter can, and verify the corrected bundled setup with the same commands.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.