Python3 C++ Module – Exception – GIL not held

  c++, cmake, module, python-3.x, windows

I have created a minimal reproduction of an issue I am having, trying to create a C++ based Python3 module. The build environment is CMake, Visual Studio Pro 2019, WinSDK 10.0.18362, Python 3.9.4. When executing:

python3 -c "import pymod"

I will get an exception in release mode with a NULL access. In debug Python, I get additional information.

My CMakeLists.txt

cmake_minimum_required(VERSION 3.15)

project(pymod_minimal_repo_example)

find_package(Python3 COMPONENTS Interpreter Development)

add_library(pymod SHARED)
target_sources(pymod PRIVATE pymodmodule.cpp)
set_target_properties(pymod PROPERTIES SUFFIX ".pyd")
target_compile_options(pymod PRIVATE /Zi)
target_link_options(pymod PRIVATE /DEBUG:FULL)
target_link_libraries(pymod PRIVATE ${Python3_LIBRARIES})
target_include_directories(pymod PRIVATE ${Python3_INCLUDE_DIRS})

Following this python reference: https://docs.python.org/3.9/extending/extending.html#a-simple-example I created the following:

#define PY_SSIZE_T_CLEAN
#include <Python.h>

static struct PyModuleDef pymodmodule = {
    PyModuleDef_HEAD_INIT, // m_base
    "pymod",               // m_name
    NULL,                  // m_doc
    -1,                    // m_size - submod not support must be static struct
    NULL,                  // m_methods - no functions present
    NULL,                  // m_slots - must be NULL
    NULL,                  // m_traverse - not needed
    NULL,                  // m_clear - not needed
    NULL                   // m_free - not needed
};

PyMODINIT_FUNC PyInit_pymod(void) {
  return PyModule_Create(&pymodmodule);
}

It is a functionally void module which does nothing, but should still run. Note that having m_methods defined, produces the same failure.

When the failure occurs, the following is output to the console:

E:projectspymodbuildDebug>python -c "import pymod"
Fatal Python error: _PyInterpreterState_GET: the function must be called with the GIL held, but the GIL is released (the current Python thread state is NULL)
Python runtime state: unknown

WinDbg jit debugger then catches the issue. A partial call stack shows my PyInit_pymod is called and when creating the python module, it cascades to a failure:

0:000> k
 # Child-SP          RetAddr           Call Site
00 000000dc`abfed4a8 00007fff`71d51385 KERNELBASE!wil::details::DebugBreak+0x2
01 000000dc`abfed4b0 00007fff`71d511a8 python39_d!fatal_error_exit+0x15 [D:asPythonpylifecycle.c @ 2201] 
02 000000dc`abfed4e0 00007fff`71d4ea98 python39_d!fatal_error+0x1b8 [D:asPythonpylifecycle.c @ 2286] 
03 000000dc`abfed540 00007fff`71cbe730 python39_d!_Py_FatalErrorFunc+0x38 [D:asPythonpylifecycle.c @ 2302] 
04 000000dc`abfed580 00007fff`71aa8044 python39_d!_Py_FatalError_TstateNULL+0x10 [D:asPythonceval.c @ 251] 
05 000000dc`abfed5b0 00007fff`71aa6cf2 python39_d!_PyInterpreterState_GET+0x34 [D:asIncludeinternalpycore_pystate.h @ 105] 
06 000000dc`abfed5f0 00007fff`d0be13e7 python39_d!PyModule_Create2+0x12 [D:asObjectsmoduleobject.c @ 168] 

>>>> 07 000000dc`abfed620 00007fff`751165dc pymod!PyInit_pymod+0x27 [E:projectspymodpymodmodule.cpp @ 19]

08 000000dc`abfed660 00007fff`751167ee python39!_PyImport_LoadDynamicModuleWithSpec+0x104 [C:AsPythonimportdl.c @ 165] 
09 000000dc`abfed6d0 00007fff`75116749 python39!_imp_create_dynamic_impl+0x86 [C:AsPythonimport.c @ 2299] 
0a 000000dc`abfed700 00007fff`750cb94b python39!_imp_create_dynamic+0x39 [C:AsPythonclinicimport.c.h @ 330] 
0b 000000dc`abfed730 00007fff`750ad500 python39!cfunction_vectorcall_FASTCALL+0x9b [C:AsObjectsmethodobject.c @ 426] 
0c 000000dc`abfed7a0 00007fff`750ad2ef python39!PyVectorcall_Call+0x5c [C:AsObjectscall.c @ 248] 
0d 000000dc`abfed800 00007fff`750ad418 python39!_PyObject_Call+0x4f [C:AsObjectscall.c @ 287] 
0e (Inline Function) --------`-------- python39!PyObject_Call+0xc [C:AsObjectscall.c @ 293] 
0f 000000dc`abfed830 00007fff`7508c65f python39!do_call_core+0xb8 [C:AsPythonceval.c @ 5092] 
10 000000dc`abfed880 00007fff`75083963 python39!_PyEval_EvalFrameDefault+0x5d6f [C:AsPythonceval.c @ 3581] 
11 (Inline Function) --------`-------- python39!_PyEval_EvalFrame+0x13 [C:AsIncludeinternalpycore_ceval.h @ 40] 
12 000000dc`abfedbb0 00007fff`750855a7 python39!_PyEval_EvalCode+0x2b3 [C:AsPythonceval.c @ 4327] 
13 000000dc`abfedc80 00007fff`7508823d python39!_PyFunction_Vectorcall+0x257 [C:AsObjectscall.c @ 396] 
14 000000dc`abfedd80 00007fff`7508812f python39!_PyEval_EvalFrameDefault+0x194d [C:AsPythonceval.c @ 3487] 
15 000000dc`abfee0b0 00007fff`750888e5 python39!_PyEval_EvalFrameDefault+0x183f [C:AsPythonceval.c @ 3504] 
16 000000dc`abfee3e0 00007fff`750888e5 python39!_PyEval_EvalFrameDefault+0x1ff5 [C:AsPythonceval.c @ 3518] 
17 000000dc`abfee710 00007fff`750888e5 python39!_PyEval_EvalFrameDefault+0x1ff5 [C:AsPythonceval.c @ 3518] 
18 000000dc`abfeea40 00007fff`750854c4 python39!_PyEval_EvalFrameDefault+0x1ff5 [C:AsPythonceval.c @ 3518] 

I cannot find any information on the console message, nor the int3. I have tried full purge and reinstall/update of Python.

Can anyone offer some direction to help, or know the cause?

Source: Windows Questions

One Reply to “Python3 C++ Module – Exception – GIL not held”

  • You can try below define _DEBUG in you module for Debug mode.

    #define PY_SSIZE_T_CLEAN

    #ifdef _DEBUG
    #undef _DEBUG
    #include
    #define _DEBUG
    #else
    #include
    #endif

LEAVE A COMMENT