distutils.filelist.findall falls into infinite loop over a symlink to a parent directory

Originally reported by: xi (Bitbucket: xi, GitHub: xi)

In pull request #105, a patch to distutils.filelist.findall() was updated to have it follow symlinks. Unfortunately, it now falls into an infinite loop over a symlink to a parent directory (we use such links to maintain a commonjs dependency graph). It could be easily fixed by eliminating already seen directory entries in findall(). Here is a patch:

diff -r 4b5954d5e760 setuptools/__init__.py
--- a/setuptools/__init__.py    Sat Jan 17 21:41:55 2015 +0100
+++ b/setuptools/__init__.py    Sun Jan 18 10:15:11 2015 -0500
@@ -137,8 +137,14 @@
     """Find all files under 'dir' and return the list of full filenames
     (relative to 'dir').
     """
+    seen = set()
     all_files = []
     for base, dirs, files in os.walk(dir, followlinks=True):
+        seen.add(os.path.realpath(base))
+        for dir in dirs[:]:
+            realpath = os.path.realpath(os.path.join(base, dir))
+            if realpath in seen:
+                dirs.remove(dir)
         if base==os.curdir or base.startswith(os.curdir+os.sep):
             base = base[2:]
         if base:

• Bitbucket: https://bitbucket.org/pypa/setuptools/issue/332

Read setuptools/init.py at distutils.filelist.findall and inspect how its os.walk traversal handles symlinked directories. Reproduce the parent-directory symlink case described in the issue, then verify that traversal terminates without repeatedly visiting an already seen directory while still finding the expected files.