PyCQA / PyCQA/isort

Sorting behavior depends on the order of passed files

Open
#1,907 0 comments 7 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
7k
Forks
687
Avg merge
4h 56m
Merged PRs (30d)
2

Description

Given this repo, when called with isort apps/b/main.py --verbose, isort classifies main.py's imports as following:

else-type place_module for os returned STDLIB
else-type place_module for starlette returned THIRDPARTY
from-type place_module for sdk returned FIRSTPARTY
from-type place_module for src.config returned LOCALFOLDER

This matches the isort configuration section in the corresponding pyproject.toml. So far so good.

Invoking isort from the same directory with isort **/*.py --verbose, however, gives us quite different, and unexpected output:

else-type place_module for os returned STDLIB
else-type place_module for starlette returned THIRDPARTY
from-type place_module for sdk returned THIRDPARTY
from-type place_module for src.config returned THIRDPARTY

It looks like in this invocation, main.py's co-located pyproject.toml was effectively ignored.

Now, the tricky bit that makes this hard to reproduce:

  • This difference in behavior seems to be triggered by a presence of another Python file outside of the apps/b directory which contains pyproject.toml.
  • To trigger the bug, the other Python file needs to come first in the directory iteration order.

In the linked repo, that file is apps/a/other.py. Note that there is also apps/c/another.py, which presence is inconsequential. Removing apps/a/other.py from the repo, or moving it to, say, apps/d/other.py, fixes the discrepancy between the two invocations.

Similarly, when explicitly provided more than one input file through the command line, the results could differ depending on the order of these files:

Expected:

$ isort apps/b/main.py apps/a/other.py --verbose

else-type place_module for os returned STDLIB
else-type place_module for starlette returned THIRDPARTY
from-type place_module for sdk returned FIRSTPARTY
from-type place_module for src.config returned LOCALFOLDER
else-type place_module for os returned STDLIB

Uh-oh:

$ isort apps/a/other.py apps/b/main.py --verbose

else-type place_module for os returned STDLIB
else-type place_module for os returned STDLIB
else-type place_module for starlette returned THIRDPARTY
from-type place_module for sdk returned THIRDPARTY
from-type place_module for src.config returned THIRDPARTY

In addition to making you question your sanity :), this semi-nondeterministic behavior can be particularly problematic in the context of precommit checks that run on a subset of repo's files: depending on the files touched, your imports' order can fluctuate from one commit to another.

Reproduced on isort 5.10.1.

Potentially related: https://github.com/PyCQA/isort/issues/1890, https://github.com/PyCQA/isort/issues/1872.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the issue with the linked isort-monorepo-repro files, especially apps/a/other.py, apps/b/main.py, and their pyproject.toml, using the two isort command forms and both file orders. Trace how configuration is selected for each input file; done means import classification is consistent regardless of directory iteration or command-line file order.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.