Skip to content

gh-140729: profiling or tracing multiprocessing can cause error #141874

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
yihong0618 wants to merge 6 commits into
base: main
Choose a base branch
from
+39 −1
Open

gh-140729: profiling or tracing multiprocessing can cause error #141874

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
4250035
fix: profiling or tracing multiprocessing can cause error
yihong0618 Nov 23, 2025
c217303
fix: add news
yihong0618 Nov 23, 2025
d4f4615
fix: news format
yihong0618 Nov 23, 2025
50dec2f
fix: skip these tests on windows
yihong0618 Nov 23, 2025
a8a26da
fix: only fix cProfile
yihong0618 Dec 3, 2025
dbf6e55
fix: forget to delete lines
yihong0618 Dec 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Lib/profiling/tracing/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -197,7 +197,7 @@ def main():
# in the module's namespace.
globs = module.__dict__
globs.update({
'__spec__': spec,
Copy link
Contributor

@YvesDup YvesDup Dec 1, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By removing spec value to the __spec__key, does the comment above still apppropriate ?
Should not we replace it here with a reference to the issue and @gaogaotiantian original fix ?

Copy link
Member

@gaogaotiantian gaogaotiantian Dec 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So my original "source" is https://docs.python.org/3/reference/import.html#module-specs - where it says the __main__ module normally sets __spec__ to None. But it's for my individual project so as long as it works it's fine. Not sure about the very accurate and official answer.

Copy link
Contributor Author

@yihong0618 yihong0618 Dec 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By removing spec value to the __spec__key, does the comment above still apppropriate ? Should not we replace it here with a reference to the issue and @gaogaotiantian original fix ?

remove is not right still fail...we the right is set it to None

Copy link
Contributor

@YvesDup YvesDup Dec 3, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove is not right still fail...we the right is set it to None

You are right.

How about adding a comment such as:
See gh-140729: set None to __spec__ according to the documentation (https://docs.python.org/3/reference/import.html#module-specs)

Copy link
Contributor Author

@yihong0618 yihong0618 Dec 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the original fixed from gaotian do not add it, I follow it style~

Copy link
Contributor

@YvesDup YvesDup Dec 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Such a shame, add a comment might be helpful.

'__spec__': None,
'__file__': spec.origin,
'__name__': spec.name,
'__package__': None,
Expand Down
37 changes: 37 additions & 0 deletions Lib/test/test_profiling/test_tracing_profiler.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,14 @@

import sys
import unittest
import subprocess

# rip off all interesting stuff from test_profile
import profiling.tracing as cProfile
import tempfile
import textwrap
from test.test_profile import ProfileTest, regenerate_expected_output
from test.support import script_helper, os_helper, SHORT_TIMEOUT
from test.support.script_helper import assert_python_failure, assert_python_ok
from test import support

Expand Down Expand Up @@ -170,6 +172,41 @@ class Foo:
f.close()
assert_python_ok('-m', "cProfile", f.name)

@unittest.skipIf(sys.platform == 'win32',
'Profiler with multiprocessing can not run on win32')
def test_profile_multiprocessing(self):
test_script = '''
import multiprocessing

def worker_proc(x):
return x * 42

def main_proc():
p = multiprocessing.Process(target=worker_proc, args=(10,))
p.start()
p.join()
print("SUCCESS")

if __name__ == "__main__":
main_proc()
'''
with os_helper.temp_dir() as temp_dir:
script = script_helper.make_script(
temp_dir, 'test_cprofile_multiprocessing', test_script
)
with script_helper.spawn_python(
"-m", "cProfile",
script,
stderr=subprocess.PIPE,
text=True
) as proc:
proc.wait(timeout=SHORT_TIMEOUT)
stdout = proc.stdout.read()
stderr = proc.stderr.read()

self.assertIn("SUCCESS", stdout)
self.assertNotIn("has no attribute \'worker_proc\'", stderr)


def main():
if '-r' not in sys.argv:
Expand Down
1 change: 1 addition & 0 deletions Misc/NEWS.d/next/Library/2025-11-23-19-06-41.gh-issue-140729.bk_HDs.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Set ``__main__.__spec__`` to ``None`` when running a script with :mod:`cProfile`
Loading