python / python/cpython

Improve Testing For Importing Multiple Extension Modules From One File (and Deal With _testimportmultiple.c)

未关闭
#124,978 2 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

还没有人认领这个 Issue。

3.14 extension-modules tests
主要语言
Python
星标
77.2k
派生
35.9k
PR 合并指标
PR 指标待抓取

描述

(low priority)

The capability of loading multiple extension modules from a single .so file was added over a decade ago. AFAIK the feature wasn't actually tested at all for several years. The PEP 489 implementation added some test coverage, but only indirectly, superficially, and (perhaps) unintentionally. Even then, the feature wasn't tested for single-phase init modules until a couple years ago and, again, only indirectly, superficially, and unintentionally. (FWIW, I'm not sure the feature is meaningfully documented at all.)

Here's the related history:

(expand)
  • 2012 (3.4) - Modules/_testimportmultiple.c added (6b2cbeba58aeb0755bdefd02fb51a80ec66d6144); not actually used?
  • 2015 (3.5) - Modules/_testmultiphase.c added for PEP 489 (d5cacbb1d9c3edc02bf0ba01702e7c06da5bc318); contained multiple modules (like `_testimportmultiple), but actually used in tests
  • 2022 (3.12) - Modules/_testsinglephase.c added (gh-99039); so we could stop using the _testcapi module to exercise singlephase init in tests; contained the impl. of the one module
  • 2023 (3.12) - Modules/_testsinglephase.c now implements multiple extension modules (gh-101891)

At the time we added the feature originally, we also added an example implementation: Modules/_testimportmultiple.c. However, it looks like it hasn't ever actually been used in any tests. (Perhaps I'm missing something.) We should use it and its modules to test the feature, or drop the file.

With all that in mind, we should consider improving test coverage of the feature, and use _testimportmultiple.c (or drop it).

To be clear, sorting this out is fairly low priority. We have basic coverage of the feature, I'm not sure it's very widely used, and the status quo isn't causing any problems.

(Honestly, I've created this issue only because I noticed _testimportmultiple.c isn't used anywhere and because of how clunky tests using _testsinglephae.c and _testmultiphase.c are to both understand and to add. However, while looking closer, I realized the feature wasn't explicitly tested, nor thoroughly.)


Here's how I think we should proceed:

  1. determine what aspects of the feature need to be tested (see below)
  2. create the new Modules/_testimportmultiple directory
  3. for each test case:
    a. implement the test (e.g. in Lib/test/test_import/__init__.py or Lib/test/test_import/test_ext_multiple_in_file.py)
    b. add the new .c file (see the devguide) under Modules/_testimportmultiple/
    c. implement the relevant modules there
  4. delete Modules/_testimportmultiple.c

We should be sure the following is tested:

  • multiphase-init
    • 2 modules in the same file (1 direct, 1 indirect)
      • load direct only
      • load indirect only
      • load direct then indirect
      • load indirect then direct
    • 3 modules in the same file (1 direct, 2 indirect)
      • similar permutations, including both indirect-only, indirect-indirect-only, indirect-direct-indirect, etc.
    • 1 indirect module without a direct one (init func name does not match filename
  • singlephase-init
    • same as multi-phase init
    • 2 modules in same file that share some non-exported state stored in a global variable
  • singlephase-init and multiphase-init modules in the same file

We should be sure that the new tests include whatever _testsinglephase.c and _testmultiphase.c currently cover specific to multiple-modules-in-a-file.

The modules for each case could all be in a single file, i.e. the existing Modules/_testimportmultiple.c, but it probably makes more sense to implement each case in its own file (in a new Modules/_testimportmultiple directory).


Again, the feature is actually tested currently, albeit indirectly and only superficially, through Modules/_testmultiphase.c and Modules/_testsinglephase.c. However, as far as I know, there isn't any need for the multiple-modules-in-one-file approach in either case. Once we're directly testing the feature using _testimportmultiple.c, we can stop testing it indirectly with the other two. See gh-124983.

贡献指南

打开贡献指南

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

调研方向

从 Modules/_testimportmultiple.c、Modules/_testsinglephase.c、Modules/_testmultiphase.c 以及 Lib/test/test_import 下的现有测试开始。查看扩展模块的开发者工作流指南,然后确定所需的直接覆盖范围,以及是否应复用或删除旧的 C 文件。完成的标准是,显式测试覆盖所列出的单阶段、多阶段、混合和共享状态情况。

由索引模型根据 Issue 内容生成。

评估

技术栈
c, python
领域
testing
Issue 类型
重构
难度
4/5
预计耗时
3-5 天
活跃度
停滞
描述清晰度
基本清楚
新手友好度
35/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。