python / python/cpython

Revisit adding lexical normalization support to `pathlib`

未關閉
#124,825 11 則留言 10 個 reaction 已指派 0 人 在 GitHub 檢視

還沒有人認領這個 Issue。

stdlib topic-pathlib type-feature
主要語言
Python
星號
77.2k
分支
36k
PR 合併指標
PR 指標待擷取

描述

Feature or enhancement

Proposal:

I'd like to add a resolve_lexical method to concrete pathlib objects:

def resolve_lexical(self, /, strict=False):
    """Make the path absolute, and also normalize it, *without* resolving symlinks."""

As with resolve(), if strict is True and any segment of the given path doesn't exist, FileNotFoundError is raised (so /some/dir/_nonexistent_/.. would fail). If strict is False (the default), all path segments are processed without checking whether they exist (so /some/dir/_nonexistent_/.. lexically resolves to /some/dir).

While theoretically this could be added to PurePath without the strict option, I don't see any significant benefit to that (whereas I do see benefits to paralleling the Path.resolve() API as closely as possible).

As a minor note, adding this method would give a more direct way of checking if a path contains any symlinks at any level: path.resolve() == path.resolve_lexical() (vs the current path.is_symlink() or any(segment.is_symlink() for segment in path.parents)).

Chaining the two resolution methods would also be valid (path.resolve().resolve_lexical()), with symlinks then being resolved in the segments that actually exist, and the rest of the path, if any, being resolved lexically)


On my current project, I recently ran into a pair of subtle symlink-and-relative-path-handling bugs.

  • the original code used path.resolve() to fully resolve a path to its actual target. This gave a dynamic loading error on macOS, because one of the dynamic libraries the referenced executable needed was stored relative to the symlink, not relative to the actual binary. While that feels like a bug in the way the offending executable was packaged, handling it meant actively avoiding fully resolving paths, and instead respecting their nominal locations.
  • switching to path.absolute() not only turned off the symlink resolution, it also turned off the path normalisation that removes \..\ segments. This also resulted in dynamic loading errors on macOS when those segments were present in the executable reference (probably due to an underlying Python or OS API conditionally doing its own equivalent of path.resolve() when /../ was present in the path, since the resulting dynamic loading errors looked very similar to those I saw when hitting the first bug).

The resolution that handled both situations ended up being to use os.path to perform lexical normalisation (via os.path.abspath):

import os
import os.path

from pathlib import Path

def as_normalized_path(path:str|os.PathLike[str], /) -> Path:
    """Normalize given path and make it absolute, *without* resolving symlinks

    Expands user directory references, but *not* environment variable references.
    """
    # Ensure user directory references are handled as absolute paths
    expanded_path = os.path.expanduser(path)
    return Path(os.path.abspath(expanded_path))

Having to drop down to the lower level API to request "resolve /../ relative to the path as given" instead of the default "resolve /../ relative to symlink targets" feels like an unnecessary gap in the abstraction layer.

Has this already been discussed elsewhere?

This is a minor feature, which does not need previous discussion elsewhere

Links to previous discussion of this feature:

Previously suggested here: https://github.com/python/cpython/issues/83105

When it comes to symlink security vulnerabilities arising from parent directory traversal, they mostly relate to using symlink resolution to reach an unexpected location:

  • a human reading /some/dir/symlink/../sibling will expect it to refer to /some/dir/sibling (lexical normalization)
  • Path(/some/dir/symlink/../sibling).resolve() actually refers to /parent_of_symlink_target/sibling (i.e. you have no idea where it points without access to the filesystem state that specifies the destination of /some/dir/symlink)

As a result, the rationale for rejection doesn't feel strong to me (since the intuitive behaviour is unavailable in the high level API, and instead only the subtle system state dependent behaviour is offered)

Also noting that Java does offer lexical normalisation on its Path abstraction: https://docs.oracle.com/en/java/javase/23/docs/api/java.base/java/nio/file/Path.html#normalize()

Linked PRs
  • gh-135406

貢獻指南

開啟貢獻指南

從這裡開始

  1. 先讀完整個 Issue,再讀專案的貢獻指南。
  2. 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
  3. Fork 儲存庫,在一個分支上完成修改。
  4. 送出 Pull Request,並在描述裡引用這個 Issue 編號。

研究方向

先從 pathlib 的具體 Path API 以及提案中描述的現有 Path.resolve() 行為開始,然後比較 os.path.abspath() 的詞法正規化行為。在繼續之前,審閱連結的 PR gh-135406 和先前的討論;要視為完成,需要一個達成共識的 resolve_lexical API、strict 模式語意以及相應的 pathlib 測試。

由索引模型根據 Issue 內容生成。

評估

技術堆疊
python
領域
operating-systems
Issue 類型
功能
難度
5/5
預估耗時
一週以上
活躍度
停滯
描述清晰度
描述清楚
新手友好度
25/100

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。