-
Notifications
You must be signed in to change notification settings - Fork 2.2k
Description
Required prerequisites
- Make sure you've read the documentation. Your issue may be addressed there.
- Search the issue tracker and Discussions to verify that this hasn't already been reported. +1 or comment there if it has.
- Consider asking first in the Gitter chat room or in a Discussion.
What version (or hash if on master) of pybind11 are you using?
3.0.1
Problem description
- i want use subinterpreter in multithread
CPython version 3.14
pybind11 3.0.1
this is my c++ code
`
PYBIND11_EMBEDDED_MODULE(printer, m, py::multiple_interpreters::per_interpreter_gil()) {
m.def("which", [](const std::string& when) {
std::cout << when << "; Current Interpreter is "
<< py::subinterpreter::current().id()
<< std::endl;
});
}
void worker(int id)
{
try {
// 在 worker 线程内 创建子解释器(必须在该线程销毁)
py::subinterpreter sub = py::subinterpreter::create();
// 在激活子解释器并持有对应 GIL 的作用域内运行所有 Python 调用
{
py::subinterpreter_scoped_activate activate(sub);
py::gil_scoped_acquire gil; // 必须持有 GIL 才能安全调用 Python
try {
// 直接 import 并使用模块 —— 不要把 module 对象带出此作用域
py::module_::import("printer").attr("which")(
std::string("worker ") + std::to_string(id) + " : activated subinterpreter"
);
py::module::import("numpy");
printf("========current id:%d", id);
py::exec(R"(
import sys
print("=== sys.path ===")
for p in sys.path:
print(p)
)");
// 再做一次示例调用(同在激活+GIL作用域内)
py::module_::import("printer").attr("which")(
std::string("worker ") + std::to_string(id) + " : second call"
);
// 演示抛出 Python 异常并安全捕获(可选)
// try {
// py::module_::import("nonexistent_module"); // 会抛 error_already_set
// } catch (const py::error_already_set &e) {
// std::string msg = e.what(); // safe: 在 GIL + sub interpreter scope
// std::cerr << "Caught python error in worker " << id << ": " << msg << '\n';
// }
}
catch (const py::error_already_set &e) {
// 这里仍然在 sub interpreter + GIL 的作用域内,所以可以安全地调用 e.what()
std::string msg = e.what();
std::cerr << "Python exception inside worker " << id << ": " << msg << std::endl;
// 将 Python 异常信息转成纯 C++ 异常或返回码
}
// gil 离开作用域后不能再操作任何 Python 对象
}
// sub 在离开作用域后,在同一线程析构 —— 满足 Python 3.12 的 destroy-on-creator-thread 要求
}
catch (const std::exception &ex) {
std::cerr << "Worker " << id << " std::exception: " << ex.what() << std::endl;
}
}
int test_main_thread()
{
py::scoped_interpreter guard{};
// 在主线程里可以做一些主解释器的调用(如果需要)
{
py::gil_scoped_acquire gil;
py::module_::import("printer").attr("which")("main: before starting worker threads");
}
py::gil_scoped_release release;
std::thread t1(worker, 1);
std::thread t2(worker, 2);
t1.join();
t2.join();
}`
printer can excute but numpy give me such tips
Python exception inside worker 1: ModuleNotFoundError: No module named 'numpy'
Python exception inside worker 2: ModuleNotFoundError: No module named 'numpy'
when i use these code
py::exec(R"(
import sys
sys.path.append(r"C:\Users\shida\AppData\Local\Programs\Python\Python312\Lib")
sys.path.append(r"C:\Users\shida\AppData\Local\Programs\Python\Python312\Lib\site-packages")
)");
append the numpy lib in sys.path give me can't use in source directory
Reproducible example code
Is this a regression? Put the last known working version here if it is.
Not a regression