Document how import chains are affected by `sys.setrecursionlimit`
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 77.2k
- Forks
- 35.9k
- PR merge metrics
- PR metrics pending
Description
While working on a very large Python codebase, I ran into some undocumented behaviour.
A chain of imports (i.e. one module imports another, which imports another, which imports another) will cause a RecursionError to be raised if the Python recursion limit is not high enough. (Note, there is no recursion involved.)
Longer chains of imports can be enabled by increasing the recursion limit. But the limit has no effect beyond a certain point (around 10,000). (I believe this is due to Py_C_RECURSION_LIMIT, mentioned in by Serhiy Storchaka.)
A workaround is available: you can import the modules in a different order.
I've created a repository with more analysis and scripts to reproduce the problem. I also started a Discuss topic on the subject.
From what I understand, this is a complex problem that is unlikely to be fixed. But I think it would be helpful to document this, probably in the docs for sys.setrecursionlimit.
I would be happy to try to do a pull request, assuming the core team agree that it should be added.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the sys.setrecursionlimit documentation and use the linked import-recursion-demo repository and its reproduction scripts to verify the behavior. Done means documenting how long import chains trigger RecursionError, the approximate upper limit, and the alternate import-order workaround.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- documentation
- Issue type
- Documentation
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 58/100