bdist_egg --exclude-source-files does not work with Python3
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2.9k
- Forks
- 1.4k
- Avg merge
- 1d 1h
- Merged PRs (30d)
- 1
Description
(Coming from here: https://bugs.python.org/issue28630)
I'm trying to prepare an egg file without source code.
I had used python setup.py bdist_egg --exclude-source-files, which does not work with Python3.
After digging into the issue more, I found PEP-3147 is related to the problem.
https://www.python.org/dev/peps/pep-3147/
$ git clone https://github.com/dmiyakawa/python_greeting.git
..
$ cd python_greeting
$ python --version
Python 3.5.2
$ python setup.py bdist_egg --exclude-source-files
..
$ unzip -l dist/greeting-1.0.0-py3.5.egg
Archive: dist/greeting-1.0.0-py3.5.egg
Length Date Time Name
--------- ---------- ----- ----
1 11-09-2016 16:31 EGG-INFO/dependency_links.txt
57 11-09-2016 16:31 EGG-INFO/entry_points.txt
319 11-09-2016 16:31 EGG-INFO/PKG-INFO
213 11-09-2016 16:31 EGG-INFO/SOURCES.txt
9 11-09-2016 16:31 EGG-INFO/top_level.txt
1 11-09-2016 16:31 EGG-INFO/zip-safe
139 11-09-2016 16:31 greeting/__pycache__/__init__.cpython-35.pyc
392 11-09-2016 16:31 greeting/__pycache__/greeting.cpython-35.pyc
--------- -------
1131 8 files
Here, greeting/__pycache__/__init__.cpython-35.pyc and greeting/__pycache__/greeting.cpython-35.pyc are not where they should be.
See the following results (with Python 2.7.12)
$ python --version
Python 2.7.12
$ python setup.py bdist_egg --exclude-source-files
..
$ unzip -l dist/greeting-1.0.0-py2.7.egg
Archive: dist/greeting-1.0.0-py2.7.egg
Length Date Time Name
--------- ---------- ----- ----
1 11-09-2016 16:32 EGG-INFO/dependency_links.txt
57 11-09-2016 16:32 EGG-INFO/entry_points.txt
319 11-09-2016 16:32 EGG-INFO/PKG-INFO
213 11-09-2016 16:32 EGG-INFO/SOURCES.txt
9 11-09-2016 16:32 EGG-INFO/top_level.txt
1 11-09-2016 16:32 EGG-INFO/zip-safe
143 11-09-2016 16:32 greeting/__init__.pyc
482 11-09-2016 16:32 greeting/greeting.pyc
--------- -------
1225 8 files
According to the chart in PEP-3147, original py file must also exist to make pyc cache in __pycache__ effective.
To let bdist_egg's --exclude-sourcce-files option work as it worked in Python2 era, pyc files in __pycache__ must be at where legacy Python2 placed. I tried tweaking the Py3's egg content with the following code, and the tweaked egg worked nicely.
RE_PYCACHE_FILE = re.compile('(.+/)__pycache__/(.+)\.cpython-3\d(.pyc)$')
with tempfile.TemporaryDirectory() as tmpdir:
names_lst = []
tmpdir_abs_path = os.path.abspath(tmpdir)
with ZipFile(egg_path, 'r') as zf:
for name in zf.namelist():
m = RE_PYCACHE_FILE.match(name)
if m:
w_name = m.group(1) + m.group(2) + m.group(3)
else:
w_name = name
w_path = os.path.join(tmpdir_abs_path, w_name)
dir_path = os.path.dirname(w_path)
if not os.path.exists(dir_path):
os.makedirs(dir_path)
with open(w_path, 'wb') as file_to_write:
with zf.open(name) as file_to_read:
file_to_write.write(file_to_read.read())
names_lst.append(w_name)
with ZipFile(egg_path, 'w', compression=ZIP_DEFLATED) as zf:
for name in names_lst:
r_path = os.path.join(tmpdir_abs_path, name)
zf.write(r_path, arcname=name)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the bdist_egg entry point and reproduce the issue with python setup.py bdist_egg --exclude-source-files using the Python 3 example. Compare the generated archive with the Python 2 listing and review PEP 3147's cache-path rules. Done means Python 3 eggs exclude source files while placing bytecode where they remain usable.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- build-system
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100