gh-142155: Fix infinite recursion in shutil.copytree on Windows junctions #142156

ChuheLin · 2025-12-01T17:12:13Z

Issue: shutil.copytree infinite recursion crash on Windows Directory Junctions (WinError 206) #142155

Summary

This PR fixes a critical crash (WinError 206 / WinError 1921) in shutil.copytree on Windows when encountering directory junctions that point to a parent directory (recursive cycles).

The Problem

On Windows, shutil.copytree contains specific logic to force traversal into Directory Junctions by treating them as standard directories (is_symlink is forced to False).

However, unlike os.walk or shutil.rmtree (after recent fixes), copytree lacked a cycle detection mechanism for these traversed junctions. This caused infinite recursion until the path exceeded the OS limit, resulting in an unhandled OSError (Crash) rather than a Python RecursionError.

The Fix

I implemented cycle detection using file system identity (st_dev, st_ino), similar to how deepcopy handles recursion:

Added a hidden _seen set parameter to copytree and its helper _copytree.
Before entering a directory, the code now checks if (st.st_dev, st.st_ino) is already in _seen.
If a cycle is detected, it raises a shutil.Error("Infinite recursion detected") immediately, preventing the crash.

Verification

Regression Test: Added test_copytree_recursive_junction to Lib/test/test_shutil.py.
- Status: PASSED on local Windows build.
Manual Verification: Verified with a reproduction script that previously caused a WinError 206 crash.
- Before fix: Script crashed after creating a path > 4000 chars.
- After fix: Script correctly raises shutil.Error: Infinite recursion detected.

… junctions

bedevere-app · 2025-12-01T17:12:17Z

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

aisk · 2025-12-03T16:33:20Z

Lib/shutil.py


 def copytree(src, dst, symlinks=False, ignore=None, copy_function=copy2,
-             ignore_dangling_symlinks=False, dirs_exist_ok=False):
+             ignore_dangling_symlinks=False, dirs_exist_ok=False, _seen=None):


I'm not sure, but I think adding a private parameter to a public function is a little odd. I see we added the _seen here because _copytree calls the public copytree, so we need to add it here too.

I wonder if we could just change _copytree to call itself recursively instead of adding this private parameter. This would also avoid triggering the audit recursively (should we do that?).

If we should trigger the audit recursively, maybe we could create another private function for the recursive logic and let _copytree call it, with a public wrapper that keeps the original function signature (without adding the _seen).

I agree. It is better to have a private helper for that.

picnixz

If you want to have a recursive guard, use a frozenset() as a default and update the frozenset every time. We do this in typing.py I think or maybe elsewhere (search for recursive_guard in Lib)

picnixz · 2025-12-03T17:48:18Z

Lib/shutil.py

+    if _seen is None:
+        _seen = set()
+    src_st = os.stat(src)
+    src_id = (src_st.st_dev, src_st.st_ino)
+    if src_id in _seen:
+        raise Error([(src, dst, "Infinite recursion detected")])
+    _seen = _seen.copy()
+    _seen.add(src_id)


This is costly. We do a copy every time.

Lib/test/test_shutil.py

picnixz · 2025-12-03T17:49:31Z

Lib/test/test_shutil.py

+        except Exception as e:
+            # Skip if mklink is not available or fails for any reason
+            self.skipTest(f"Failed to create junction: {e}")


I don't think it's a good idea. Can't we create a junction from Python directly? (cc @barneygale).

This is still not addressed.

Lib/test/test_shutil.py

picnixz · 2025-12-03T17:50:14Z

Lib/test/test_shutil.py

+        src_dir = os.path.join(base_dir, "Source")
+        junction_dir = os.path.join(src_dir, "Junction")
+        os.makedirs(junction_dir)


can we use lower-case letters or is it not possible for junctions?

bedevere-app · 2025-12-03T17:50:49Z

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

…d naming

Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>

picnixz

Address all changes before requesting a new review please, either by adding a comment if I was not clear or by changing the code.

picnixz · 2025-12-03T18:48:40Z

Lib/shutil.py

+    if _seen is None:
+        _seen = frozenset()


I do not understand why we do this. Why not just using a default argument? (it is no more mutable so it is fine)

picnixz · 2025-12-03T18:48:56Z

Lib/shutil.py

+    # Track visited directories to detect cycles (e.g., Windows junctions)
+    if _seen is None:
+        _seen = frozenset()
+    src_st = os.stat(src)


This call can fail and (can be) expensive. But I assume that we need it right?

picnixz · 2025-12-03T18:51:10Z

Lib/shutil.py


 def copytree(src, dst, symlinks=False, ignore=None, copy_function=copy2,
-             ignore_dangling_symlinks=False, dirs_exist_ok=False):
+             ignore_dangling_symlinks=False, dirs_exist_ok=False, _seen=None):


I agree. It is better to have a private helper for that.

picnixz · 2025-12-03T18:52:01Z

Lib/test/test_shutil.py

+        os.makedirs(junction_dir)
+
+        # Create a junction pointing to its parent, creating a cycle
+        junction_target = os.path.dirname(junction_dir)  # Points to Source


Suggested change

junction_target = os.path.dirname(junction_dir) # Points to Source

junction_target = os.path.dirname(junction_dir)

picnixz · 2025-12-03T18:52:24Z

Lib/test/test_shutil.py

+            self.skipTest(f"Failed to create junction: {e}")
+
+        # Create destination directory
+        dst_dir = os.path.join(base_dir, "Dest")


Dest -> dest

picnixz · 2025-12-03T18:52:53Z

Lib/test/test_shutil.py

+        except Exception as e:
+            # Skip if mklink is not available or fails for any reason
+            self.skipTest(f"Failed to create junction: {e}")


This is still not addressed.

ChuheLin · 2025-12-03T19:09:43Z

Address all changes before requesting a new review please, either by adding a comment if I was not clear or by changing the code.

Thanks for the detailed review! It's getting late in my time zone, so I'm going to get some rest now. I will go through all your comments and update the code first thing tomorrow.

pythongh-142155: Fix infinite recursion in shutil.copytree on Windows…

a23ffa3

… junctions

ChuheLin requested a review from giampaolo as a code owner December 1, 2025 17:12

bedevere-app bot mentioned this pull request Dec 1, 2025

shutil.copytree infinite recursion crash on Windows Directory Junctions (WinError 206) #142155

Open

bedevere-app bot added the awaiting review label Dec 1, 2025

📜🤖 Added by blurb_it.

0013914

ChuheLin changed the title ~~gh-142155: Fix infinite recursion in shutil.copytree on Windows junct…~~ gh-142155: Fix infinite recursion in shutil.copytree on Windows junctions Dec 1, 2025

ChuheLin added 2 commits December 2, 2025 11:01

Merge branch 'main' into fix-shutil-recursion-loop

cdebad2

Merge branch 'main' into fix-shutil-recursion-loop

debc710

aisk reviewed Dec 3, 2025

View reviewed changes

picnixz requested changes Dec 3, 2025

View reviewed changes

bedevere-app bot added awaiting changes and removed awaiting review labels Dec 3, 2025

ChuheLin and others added 3 commits December 4, 2025 02:32

Address review: optimize recursion with frozenset, cleanup imports an…

cee3c73

…d naming

Update Lib/test/test_shutil.py

19f55fb

Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>

Merge branch 'main' into fix-shutil-recursion-loop

d71b49b

ChuheLin requested a review from picnixz December 3, 2025 18:45

picnixz requested changes Dec 3, 2025

View reviewed changes

	junction_target = os.path.dirname(junction_dir) # Points to Source
	junction_target = os.path.dirname(junction_dir)

Uh oh!

gh-142155: Fix infinite recursion in shutil.copytree on Windows junctions #142156

Are you sure you want to change the base?

gh-142155: Fix infinite recursion in shutil.copytree on Windows junctions #142156

Conversation

ChuheLin commented Dec 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

The Problem

The Fix

Verification

Uh oh!

bedevere-app bot commented Dec 1, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

picnixz left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bedevere-app bot commented Dec 3, 2025

Uh oh!

picnixz left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ChuheLin commented Dec 3, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ChuheLin commented Dec 1, 2025 •

edited

Loading