python / python/cpython

shutil.rmtree can have O(n^2) performance on large dirs

Open
#76,634 12 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

3.7 (EOL) stdlib type-feature
Dominant language
Python
Stars
77.2k
Forks
36k
PR merge metrics
PR metrics pending

Description

BPO 32453
Nosy @pitrou, @giampaolo, @nh2, @serhiy-storchaka
Files
  • bench_rmtree.py
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = None
    created_at = <Date 2017-12-30.05:15:09.821>
    labels = ['3.7', 'type-feature', 'library']
    title = 'shutil.rmtree can have O(n^2) performance on large dirs'
    updated_at = <Date 2021-10-30.12:33:10.794>
    user = 'https://github.com/nh2'
    

    bugs.python.org fields:

    activity = <Date 2021-10-30.12:33:10.794>
    actor = 'nh2'
    assignee = 'none'
    closed = False
    closed_date = None
    closer = None
    components = ['Library (Lib)']
    creation = <Date 2017-12-30.05:15:09.821>
    creator = 'nh2'
    dependencies = []
    files = ['47355']
    hgrepos = []
    issue_num = 32453
    keywords = []
    message_count = 12.0
    messages = ['309217', '309227', '309230', '309300', '309303', '309305', '309308', '309317', '309353', '309359', '309367', '405367']
    nosy_count = 4.0
    nosy_names = ['pitrou', 'giampaolo.rodola', 'nh2', 'serhiy.storchaka']
    pr_nums = []
    priority = 'normal'
    resolution = None
    stage = None
    status = 'open'
    superseder = None
    type = 'enhancement'
    url = 'https://bugs.python.org/issue32453'
    versions = ['Python 3.7']
    

    Contributor guide

    Open the contributing guide

    First steps

    1. Read the whole issue, then the project's contributing guide.
    2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
    3. Fork the repository and make your change on a branch.
    4. Open a pull request that references the issue number.

    Research direction

    Start with the mentioned bench_rmtree.py benchmark and the shutil.rmtree implementation in the standard library. Run the benchmark on large directories to reproduce the scaling problem, then verify that the completed change removes the O(n²) behavior without breaking existing rmtree behavior.

    Written by the indexing model from the issue text.

    Assessment

    Tech stack
    python
    Domain
    operating-systems
    Issue type
    Bug
    Difficulty
    4/5
    Estimated time
    3-5 days
    Activity status
    Stale
    Clarity
    Mostly clear
    Newbie friendliness
    35/100

    Get new issues in your inbox

    A short digest of beginner-friendly GitHub issues.