Re: setup.py findFile() logic fragile to symlink loops

Robin Becker <[email protected]> Mon, 28 Mar 2022 11:26:09 +0100
Newsgroups gmane.comp.python.reportlab.user
Message-ID <[email protected]> 
Hi Michael,

can you try the attached patch. I tested locally with an artificially bad setup with plenty of loops. It failed 
initially, but worked with the new code on linux/windows10.

It fixes both findFile and listFiles.

I am isolating with a positive lateral flow today so there might be some obvious flaw.

On 28/03/2022 09:40, Michał Górny wrote:
> Hi,
> 
> I've just gotten a report from a Gentoo user that reportlab's setup.py
> hangs on build command.  After short debugging, we've discovered that
> the findFile() function used to look for freetype2 headers is fragile to
> symlink loops.
> 
........


-- 
Robin Becker
fix-symlink-loop.patch (text/x-patch, 1.6 KB) 
diff -r 57f8c398ddeb setup.py
--- a/setup.py	Fri Mar 25 10:02:37 2022 +0000
+++ b/setup.py	Mon Mar 28 10:59:21 2022 +0100
@@ -189,15 +189,40 @@
         else:
             P.insert(x, d)
 
+# protection against loops needed. reported by
+# Michał Górny &lt; mgorny at gentoo dot org &gt;
+# see https://stackoverflow.com/questions/36977259
 def findFile(root, wanted, followlinks=True):
-    for p, _, F in os.walk(root,followlinks=followlinks):
+    visited = set()
+    for p, D, F in os.walk(root,followlinks=followlinks):
+        #scan directories to check for prior visits
+        #use dev/inode to make unique key
+        SD = [].append
+        for d in D:
+            dk = os.stat(pjoin(p,d))
+            dk = dk.st_dev, dk.st_ino
+            if dk not in visited:
+                visited.add(dk)
+                SD(d)
+        D[:] = SD.__self__  #set the dirs to be scanned
         for fn in F:
             if fn==wanted:  
                 return abspath(pjoin(p,fn))
 
 def listFiles(root,followlinks=True,strJoin=None):
+    visited = set()
     R = [].append
-    for p, _, F in os.walk(root,followlinks=followlinks):
+    for p, D, F in os.walk(root,followlinks=followlinks):
+        #scan directories to check for prior visits
+        #use dev/inode to make unique key
+        SD = [].append
+        for d in D:
+            dk = os.stat(pjoin(p,d))
+            dk = dk.st_dev, dk.st_ino
+            if dk not in visited:
+                visited.add(dk)
+                SD(d)
+        D[:] = SD.__self__  #set the dirs to be scanned
         for fn in F:
             R(abspath(pjoin(p,fn)))
     R = R.__self__