gh-42948: Fix shutil.move() on permission-restricted filesystems

[ashm-dev]

• Issue: shutil.move raises OSError when copystat fails #42948

[gpshead]

As is, this also changes the default behavior of copy2 - and thus move and copytree which default to using copy2 - to not fail if the file times could not be set.

While it is arguable that the new behavior is what people might expect, raising the error in this situation has been the API for so long, we should assume someone is relying on that. (Hyrum's Law)

I think we should err on the side of caution and retain backwards compatibility but provide a way to get the error-free behavior.

At a minimum, add a keyword only argument to copystat to determine if utime permission errors should raise or not. Plumb that through into copy2 as well.

We could do a little more plumbing through to, move, and copytree - but that proliferation of arguments can get a bit gross. I suggest we just add to their docs that users who want to ignore inability to set the utime can do so via a copy_function.

With an example in Docs of passing copy_function=lambda s, d: shutil.copy2(s, d, strict_time_copy=False)

thoughts?

[picnixz]

Alternatively, we could select which stat to actually copy. We are currently calling stat, utime and chmod in copystat, so why not just changing copy2 and copystat so that we can also change which metadata to copy? more generally, we could have specify what attributes to preserve (cp preserves by default mode,ownership,timestamps which is exactly what copy2 does. But copy2 could be better and we could have

def copystat(src, dst, *, ..., preserve=("mode", "ownership", "timestamps", "links", "xattr")):
   ...

def copy2(src, dst, *, ..., preserve=("mode", "ownership", "timestamps", "links", "xattr")):
    ...
    copystat(src, dst, preserve=preserve)

and in the docs we would suggest using

copy_function = partial(copy2, preserve=("mode", "ownership", "links", "xattr"))

We could also have preserve be a flag enumeration, but this is subject to bikehseeding here (shutil is already a heavy module so it's also fine to import enum)

---

EDIT: By default we also copy xattr and links so it's likely better to have a flag. I'll open a PR.

[picnixz]

So, after playing a bit with the implementation:

• copy only preserves the mode.
• copy2 preserves mode, timestamps, and extended attributes. It preserves flags when possible (I think it's a macOS thing). It is roughly equivalent to cp --preserve=mode,timestamps,xattr + chflags call. I'm not sure we are preserving ownership (the docs say we don't).

What we want is for copy2 to possibly not preserve timestamps. But I guess the issue also exists for permission-restricted filesystems where we cannot change the extended attributes. So if we want a parameter-based approach it would be:

def copy2(src, dst, *, follow_symlinks=..., preserve_timestamps=True, preserve_xattr=True): ...

and copymetadata would also support those parameters. Ideally, I'd like to use a flag but since we are only using two parameters, it's likely easier for users to use them.

[picnixz]

Alternative implementation (without tests yet): #142098