python / python/cpython

`profile` and `cProfile` cannot profile a directory or PEP-441 zip file

未关闭
#100,961 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

还没有人认领这个 Issue。

type-bug
主要语言
Python
星标
77.2k
派生
36k
PR 合并指标
PR 指标待抓取

描述

Bug report

Python can execute zip files (PEP-441) and directories directly, e.g. python code.zip or python directory, if they're constructed appropriately (such as via zipapp or https://github.com/pantsbuild/pex). These don't work with profile or cProfile modules, e.g. python -m profile code.zip fails with ValueError: source code string cannot contain null bytes and python -m cProfile directory fails with IsADirectoryError: [Errno 21] Is a directory: 'directory'.

For example: https://gist.github.com/huonw/52574abce4057afd99abb5e4a7f82b61 has a simple __main__.py and a zip file that contains it:

import sys
print(f"Hello from {sys.argv[0]}")

There's a script that runs all 9 combinations of zip vs. file vs. directory, and no profiling vs. profile vs cProfile. It can be executed via:

git clone https://gist.github.com/huonw/52574abce4057afd99abb5e4a7f82b61
cd 52574abce4057afd99abb5e4a7f82b61
./script.sh

Output of script.sh:

updating: __main__.py (stored 0%)
Archive:  code.zip
 extracting: directory/__main__.py   
Python 3.11.0


*** Running without profiling
Hello from __main__.py
Hello from code.zip
Hello from directory


*** Running with 'profile'
Hello from __main__.py
** BUG:
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/Users/huon/.pyenv/versions/3.11.0/lib/python3.11/profile.py", line 610, in <module>
    main()
  File "/Users/huon/.pyenv/versions/3.11.0/lib/python3.11/profile.py", line 591, in main
    code = compile(fp.read(), progname, 'exec')
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: source code string cannot contain null bytes
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/Users/huon/.pyenv/versions/3.11.0/lib/python3.11/profile.py", line 610, in <module>
    main()
  File "/Users/huon/.pyenv/versions/3.11.0/lib/python3.11/profile.py", line 590, in main
    with open(progname, 'rb') as fp:
         ^^^^^^^^^^^^^^^^^^^^
IsADirectoryError: [Errno 21] Is a directory: 'directory'


*** Running with 'cProfile'
Hello from __main__.py
** BUG:
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/Users/huon/.pyenv/versions/3.11.0/lib/python3.11/cProfile.py", line 190, in <module>
    main()
  File "/Users/huon/.pyenv/versions/3.11.0/lib/python3.11/cProfile.py", line 171, in main
    code = compile(fp.read(), progname, 'exec')
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: source code string cannot contain null bytes
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/Users/huon/.pyenv/versions/3.11.0/lib/python3.11/cProfile.py", line 190, in <module>
    main()
  File "/Users/huon/.pyenv/versions/3.11.0/lib/python3.11/cProfile.py", line 170, in main
    with open(progname, 'rb') as fp:
         ^^^^^^^^^^^^^^^^^^^^
IsADirectoryError: [Errno 21] Is a directory: 'directory'

Workarounds:

  • for a zip, manually unzip, and then run the __main__.py, e.g. unzip code.zip -d directory && python -m cProfile directory/__main__.py.
  • for a directory, run the __main__.py, e.g. python -m cPorfile directory/__main__.py

Your environment

  • CPython versions tested on: 3.9.10, 3.10.4, 3.11.0
  • Operating system and architecture: python -m platform == macOS-12.5.1-arm64-arm-64bit

贡献指南

打开贡献指南

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

调研方向

从 profile.py 和 cProfile.py 的模块入口点开始,然后针对文件、zip 和目录这几种情况运行提供的复现脚本 script.sh。完成的标准是两个性能分析命令都能处理 PEP-441 zip 文件和可执行目录,且不会出现报告中的 ValueError 或 IsADirectoryError。

由索引模型根据 Issue 内容生成。

评估

技术栈
python
领域
tooling
Issue 类型
缺陷
难度
3/5
预计耗时
1-2 天
活跃度
停滞
描述清晰度
基本清楚
新手友好度
42/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。