treeverse / treeverse/dvc

check-ignore: a negation is dropped when an ancestor matches a `dir/**` pattern

Open
#11,095 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
15.9k
Forks
1.3k
PR merge metrics
No merged PRs in 30d

Description

Bug Report

Description

A ! line stops re-including when an ancestor directory is matched by a dir/** pattern. git
re-includes in that situation, because X/** excludes the contents of X and not X itself, so
the "you cannot re-include under an excluded directory" rule never fires.

Reproduce
git init -q . && dvc init -q
mkdir -p volumes/functions
touch volumes/functions/deno.json volumes/functions/other.py
printf 'volumes/functions/**\n!volumes/functions/deno.json\n' | tee .dvcignore > .gitignore

git check-ignore -q volumes/functions/deno.json && echo "git: ignored" || echo "git: NOT ignored"
dvc check-ignore -q volumes/functions/deno.json && echo "dvc: ignored" || echo "dvc: NOT ignored"
volumes/functions/deno.json      git: NOT ignored  dvc: ignored     <-- diverges
volumes/functions/other.py       git: ignored      dvc: ignored

dvc 3.67.1, pathspec 1.1.1, git 2.55.0, Python 3.14, Linux. The same verdict comes out of the
library (DvcIgnoreFilter.is_ignored_file), so it is not a check-ignore-only discrepancy like
the one discussed in #10122.

Two controls, so the variable is pinned
  • Plain negation works today. ignore.txt + !no-ignore.txt at the root: git and dvc agree,
    both APIs. Whatever #10122 was about, this is not a repeat of it.
  • Drop the ** and dvc is right. With volumes/functions/ + !volumes/functions/deno.json,
    git also refuses to re-include (the directory itself is excluded), and dvc says the same. So
    the variable is the **, not the negation.
Where it happens

DvcIgnorePatterns._ignore walks the ancestor prefixes of a path and breaks on the first one that
matches, which is the right shape for git's rule. The ancestor is tested through
_find_matching_pattern, which — for a directory — also tries path + "/":

paths = [path]
if is_dir and not path.endswith("/"):
    paths.append(f"{path}/")

pathspec compiles volumes/functions/** to ^volumes/functions/, and that regex matches the
probe string volumes/functions/. So the ancestor is judged excluded, the loop breaks at i = 2,
and the basename where !volumes/functions/deno.json would have won is never reached.

The trailing-slash probe cannot simply be dropped: a genuine directory pattern X/ compiles to
^X(?P<ps_d>/) and needs it. What is missing is the distinction between a pattern that excludes
X and one that only excludes what is under it.

Worth knowing before choosing a fix: on the same two lines,
pathspec.GitIgnoreSpec.from_lines(...).match_file("volumes/functions/deno.json") returns False
— the correct answer. The dependency you already depend on resolves this case; it is dvc's own
ancestor walk that diverges from it.

How I found it, and how rare it is

I maintain a conformance bench that compares tools against git check-ignore --no-index on rule
files harvested from real repositories, and dvc is in it because it is the one project I could find
that promises the per-directory layer rather than improvising it. Over 4441 queries from 64
repositories, this is the only divergence dvc produces. The failing case is
supabase/supabase's docker/.gitignore, which has exactly this shape:

volumes/functions/**
!volumes/functions/deno.json*

Bench, corpus and the reduction scripts: https://github.com/KaizenShogun/gitignore-conformance
happy to run any candidate fix through it.

— Midas

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at DvcIgnorePatterns._ignore and _find_matching_pattern, then reproduce the two-rule case with dvc check-ignore and compare it with git and pathspec. Preserve the trailing-slash probe needed for X/ patterns while making volumes/functions/** allow the later negation to be evaluated; verify the plain-negation and directory-pattern controls still agree.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
tooling
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.