#138473 backports to 3.13 & 3.14 break running programs on Python version upgrade (multiprocessing)

[mgorny]

Bug report

Bug description:

#138473 changed the format used to pass data between the parent process and subprocesses in multiprocessing. This change has been backported as-is to Python 3.14.1 and FWICS 3.13.10. Unfortunately, this means that if Python is upgraded to a newer patch version while applications are running using this Python version, newly started subprocesses suddenly start using the new data format that is incompatible with the parent process. I think such a change is acceptable for 3.15 now, but this is a major issue with 3.13 & 3.14, because it literally means we would have to tell our users "please restart all programs after upgrading Python, or things may silently break".

I'm sorry but I don't have a traceback handy, but it broke Gentoo's package manager for me mid-upgrade, with the parent process starting to throw JSONDecodeErrors at the non-JSON data provided by children.

Kudos to @thesamesam for finding the offending change.

CPython versions tested on:

3.13, 3.14

Operating systems tested on:

Linux

Linked PRs

• gh-142206: multiprocessing.resource_tracker: Decode messages using older protocol #142215

[gpshead]

FWIW there are a lot of problems with upgrading a Python installation across patch releases in-place that have happened over the years. I don't think this is nearly as common of a thing to do as it was 10-20y ago given the worlds use of containers.

We don't have regression tests for this concept (and the test matrix would be quite huge as it'd need to test .N to any .N+M patch upgrade).

Gentoo with I presume emerge still being a Python application (sorry it's been too long since i've run gentoo) got bitten hard I guess?

[mgorny]

Gentoo with I presume emerge still being a Python application (sorry it's been too long since i've run gentoo) got bitten hard I guess?

Yep. The package manager uses multiprocessing to install packages, so the top process couldn't communicate with running installs anymore.

[gpshead]

the old IPC message format prior to the PR that got backported was msg = f"{cmd}:{name}:{rtype}\n".encode("ascii") - we could still use that format and parse names that have ':' in them by splitting/partitioning only on the first and last colon without reinventing the format. just brainstorming; that'd probably work as a fix for 3.13 and 3.14 (no need to change main/3.15)

[eli-schwartz]

Yes, the issue as I understand it was that the communication format was changed in a point release, and the new subprocesses didn't have code to understand the old parent process.

It's a standard mechanism for compatible updates, to simply support reading both formats but only emit one of them. So that should work fine. (And something like this is useful to keep in mind for consideration whenever backporting changes into a previously released version as a bugfix.)

[encukou]

Sorry for the issue. I've sent a PR to make this more robust.

[radi2015]

An identical multi-entry concurrent data processing program, after being upgraded to Python 3.13 or 3.14, inexplicably hangs without reporting any errors.