python / python/cpython

Optimize pathlib.PurePath.relative_to walk up case

Open
#143,630 6 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

performance stdlib topic-pathlib type-feature
Dominant language
Python
Stars
77.2k
Forks
35.9k
PR merge metrics
PR metrics pending

Description

Feature or enhancement

Proposal:

pathlib.PurePath.relative_to method has a loop, which builds parents through every iteration of the loop. This makes it slow if we end up looping more than once. We could improve performance by caching the parents when looping multiple times.

The difficult part is how to make the changes so that they do not slow down two special cases:

  • path == self
  • we loop exactly once

So far all my attempts to improve this either slow down one or both of the above cases, which is why I have not yet done a PR. But if we could figure out ways to not slow down the above two, then the savings would be significant.

The best I have been able to come up with (I can make a PR if this looks good enough):

No changes if path == self
break on 1st loop, walk_up=False: 1.01x slower
1 level up: 1.09x slower
2 levels up: 1.25x faster
3 levels up: 1.46x faster
Has this already been discussed elsewhere?

This is a minor feature, which does not need previous discussion elsewhere

Links to previous discussion of this feature:

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at pathlib.PurePath.relative_to and inspect how its parent walk is built. Benchmark path == self, one-level, two-level, and three-level walk-up cases; done means improving multi-level walks without slowing the two special cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
operating-systems
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.