PyImport_Import seg. faults after calling C that calls Python, from Python












0














I am calling python code inside a C function and it works fine. However, I wanted to call the C function from python (effectively reuse all the C code) but when it comes to importing the python modules from C I get a seg. fault



(Removed all error checking from the code to make this simpler)



main.c:



#include "Python.h"

int runpython() {
Py_Initialize();
PyObject* pFunc;
PyObject *pArgs;
PyObject *pValue;
PyObject* pName = PyString_FromString("python_module");
PyObject* pModule = PyImport_Import(pName);
Py_DECREF(pModule);

pFunc = PyObject_GetAttrString(pModule, "pcall");
pArgs = PyTuple_New(1);
pValue = PyInt_FromLong(42);
PyTuple_SetItem(pArgs, 0, pValue);
pValue = PyObject_CallObject(pFunc, pArgs);
Py_DECREF(pArgs);
Py_DECREF(pValue);
Py_XDECREF(pFunc);
Py_DECREF(pModule);
}

int main() {
return runpython();
}


python_module.py



def pcall(value):
print "Value: " + str(value)


python_main.py



from ctypes import *

cdll.LoadLibrary("libMyPython.so")
libc = CDLL("libMyPython.so")
libc.runpython()


environment and execution:



export PYTHONPATH=$PYTHONPATH:$PWD
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$PWD

echo "Running C main"
gcc main.c -o main -I/usr/include/python2.7 -lpython2.7 -g
./main

echo "Running Python main"
gcc -shared -fPIC main.c -o libMyPython.so -I/usr/include/python2.7 -lpython2.7 -g
python python_main.py


A sample output is:



### Running C main
Value: 42
### Running Python main
Segmentation fault (core dumped)


Running in gdb yields the stack:



Program received signal SIGSEGV, Segmentation fault.
0x00005555556318d9 in PyImport_Import ()
(gdb) bt
#0 0x00005555556318d9 in PyImport_Import ()
#1 0x00007ffff5e07883 in runpython () at c_main.c:9
#2 0x00007ffff600edae in ffi_call_unix64 () from /usr/lib/x86_64-linux-gnu/libffi.so.6
#3 0x00007ffff600e71f in ffi_call () from /usr/lib/x86_64-linux-gnu/libffi.so.6
#4 0x00007ffff6221e56 in _ctypes_callproc ()
from /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so
#5 0x00007ffff6221505 in ?? () from /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so
#6 0x000055555564f420 in PyEval_EvalFrameEx ()
#7 0x0000555555647d0a in PyEval_EvalCodeEx ()
#8 0x0000555555647629 in PyEval_EvalCode ()
#9 0x000055555567861f in ?? ()
#10 0x0000555555673322 in PyRun_FileExFlags ()
#11 0x000055555567267d in PyRun_SimpleFileExFlags ()
#12 0x00005555556211ab in Py_Main ()
#13 0x00007ffff7a05b97 in __libc_start_main (main=0x555555620b10 <main>, argc=2,
argv=0x7fffffffde58, init=<optimised out>, fini=<optimised out>, rtld_fini=<optimised out>,
stack_end=0x7fffffffde48) at ../csu/libc-start.c:310
#14 0x0000555555620a2a in _start ()


I have tried using PyImport_ImportModule or PyRun_SimpleString("import python_module") with the same effects... Any ideas?



I am running this on an ubuntu machine using python2.7.










share|improve this question


















  • 1




    Possible duplicate of How to clear a PyList object that contains objects?
    – DavidW
    Nov 23 at 7:15






  • 1




    I know the code is different but it's the same issue - you can't be calling Python code from a CDLL ctypes object because it has released the GIL.
    – DavidW
    Nov 23 at 7:16






  • 1




    (Alternatively this answer is also the same issue, but goes into a little more detail)
    – DavidW
    Nov 23 at 7:18










  • The last answer is straight spot on thank you. I would accept it as an answer if you submit it (or should this be marked as duplicate? I can see this being useful for otherts getting the seg fault)
    – Evangelos Bempelis
    Nov 23 at 10:04










  • I don't think I can mark it as duplicate (I found it after I'd already tried to mark the first answer, and I think that stops me from suggesting another for a couple of weeks) but hopefully someone else can
    – DavidW
    Nov 23 at 10:38
















0














I am calling python code inside a C function and it works fine. However, I wanted to call the C function from python (effectively reuse all the C code) but when it comes to importing the python modules from C I get a seg. fault



(Removed all error checking from the code to make this simpler)



main.c:



#include "Python.h"

int runpython() {
Py_Initialize();
PyObject* pFunc;
PyObject *pArgs;
PyObject *pValue;
PyObject* pName = PyString_FromString("python_module");
PyObject* pModule = PyImport_Import(pName);
Py_DECREF(pModule);

pFunc = PyObject_GetAttrString(pModule, "pcall");
pArgs = PyTuple_New(1);
pValue = PyInt_FromLong(42);
PyTuple_SetItem(pArgs, 0, pValue);
pValue = PyObject_CallObject(pFunc, pArgs);
Py_DECREF(pArgs);
Py_DECREF(pValue);
Py_XDECREF(pFunc);
Py_DECREF(pModule);
}

int main() {
return runpython();
}


python_module.py



def pcall(value):
print "Value: " + str(value)


python_main.py



from ctypes import *

cdll.LoadLibrary("libMyPython.so")
libc = CDLL("libMyPython.so")
libc.runpython()


environment and execution:



export PYTHONPATH=$PYTHONPATH:$PWD
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$PWD

echo "Running C main"
gcc main.c -o main -I/usr/include/python2.7 -lpython2.7 -g
./main

echo "Running Python main"
gcc -shared -fPIC main.c -o libMyPython.so -I/usr/include/python2.7 -lpython2.7 -g
python python_main.py


A sample output is:



### Running C main
Value: 42
### Running Python main
Segmentation fault (core dumped)


Running in gdb yields the stack:



Program received signal SIGSEGV, Segmentation fault.
0x00005555556318d9 in PyImport_Import ()
(gdb) bt
#0 0x00005555556318d9 in PyImport_Import ()
#1 0x00007ffff5e07883 in runpython () at c_main.c:9
#2 0x00007ffff600edae in ffi_call_unix64 () from /usr/lib/x86_64-linux-gnu/libffi.so.6
#3 0x00007ffff600e71f in ffi_call () from /usr/lib/x86_64-linux-gnu/libffi.so.6
#4 0x00007ffff6221e56 in _ctypes_callproc ()
from /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so
#5 0x00007ffff6221505 in ?? () from /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so
#6 0x000055555564f420 in PyEval_EvalFrameEx ()
#7 0x0000555555647d0a in PyEval_EvalCodeEx ()
#8 0x0000555555647629 in PyEval_EvalCode ()
#9 0x000055555567861f in ?? ()
#10 0x0000555555673322 in PyRun_FileExFlags ()
#11 0x000055555567267d in PyRun_SimpleFileExFlags ()
#12 0x00005555556211ab in Py_Main ()
#13 0x00007ffff7a05b97 in __libc_start_main (main=0x555555620b10 <main>, argc=2,
argv=0x7fffffffde58, init=<optimised out>, fini=<optimised out>, rtld_fini=<optimised out>,
stack_end=0x7fffffffde48) at ../csu/libc-start.c:310
#14 0x0000555555620a2a in _start ()


I have tried using PyImport_ImportModule or PyRun_SimpleString("import python_module") with the same effects... Any ideas?



I am running this on an ubuntu machine using python2.7.










share|improve this question


















  • 1




    Possible duplicate of How to clear a PyList object that contains objects?
    – DavidW
    Nov 23 at 7:15






  • 1




    I know the code is different but it's the same issue - you can't be calling Python code from a CDLL ctypes object because it has released the GIL.
    – DavidW
    Nov 23 at 7:16






  • 1




    (Alternatively this answer is also the same issue, but goes into a little more detail)
    – DavidW
    Nov 23 at 7:18










  • The last answer is straight spot on thank you. I would accept it as an answer if you submit it (or should this be marked as duplicate? I can see this being useful for otherts getting the seg fault)
    – Evangelos Bempelis
    Nov 23 at 10:04










  • I don't think I can mark it as duplicate (I found it after I'd already tried to mark the first answer, and I think that stops me from suggesting another for a couple of weeks) but hopefully someone else can
    – DavidW
    Nov 23 at 10:38














0












0








0


1





I am calling python code inside a C function and it works fine. However, I wanted to call the C function from python (effectively reuse all the C code) but when it comes to importing the python modules from C I get a seg. fault



(Removed all error checking from the code to make this simpler)



main.c:



#include "Python.h"

int runpython() {
Py_Initialize();
PyObject* pFunc;
PyObject *pArgs;
PyObject *pValue;
PyObject* pName = PyString_FromString("python_module");
PyObject* pModule = PyImport_Import(pName);
Py_DECREF(pModule);

pFunc = PyObject_GetAttrString(pModule, "pcall");
pArgs = PyTuple_New(1);
pValue = PyInt_FromLong(42);
PyTuple_SetItem(pArgs, 0, pValue);
pValue = PyObject_CallObject(pFunc, pArgs);
Py_DECREF(pArgs);
Py_DECREF(pValue);
Py_XDECREF(pFunc);
Py_DECREF(pModule);
}

int main() {
return runpython();
}


python_module.py



def pcall(value):
print "Value: " + str(value)


python_main.py



from ctypes import *

cdll.LoadLibrary("libMyPython.so")
libc = CDLL("libMyPython.so")
libc.runpython()


environment and execution:



export PYTHONPATH=$PYTHONPATH:$PWD
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$PWD

echo "Running C main"
gcc main.c -o main -I/usr/include/python2.7 -lpython2.7 -g
./main

echo "Running Python main"
gcc -shared -fPIC main.c -o libMyPython.so -I/usr/include/python2.7 -lpython2.7 -g
python python_main.py


A sample output is:



### Running C main
Value: 42
### Running Python main
Segmentation fault (core dumped)


Running in gdb yields the stack:



Program received signal SIGSEGV, Segmentation fault.
0x00005555556318d9 in PyImport_Import ()
(gdb) bt
#0 0x00005555556318d9 in PyImport_Import ()
#1 0x00007ffff5e07883 in runpython () at c_main.c:9
#2 0x00007ffff600edae in ffi_call_unix64 () from /usr/lib/x86_64-linux-gnu/libffi.so.6
#3 0x00007ffff600e71f in ffi_call () from /usr/lib/x86_64-linux-gnu/libffi.so.6
#4 0x00007ffff6221e56 in _ctypes_callproc ()
from /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so
#5 0x00007ffff6221505 in ?? () from /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so
#6 0x000055555564f420 in PyEval_EvalFrameEx ()
#7 0x0000555555647d0a in PyEval_EvalCodeEx ()
#8 0x0000555555647629 in PyEval_EvalCode ()
#9 0x000055555567861f in ?? ()
#10 0x0000555555673322 in PyRun_FileExFlags ()
#11 0x000055555567267d in PyRun_SimpleFileExFlags ()
#12 0x00005555556211ab in Py_Main ()
#13 0x00007ffff7a05b97 in __libc_start_main (main=0x555555620b10 <main>, argc=2,
argv=0x7fffffffde58, init=<optimised out>, fini=<optimised out>, rtld_fini=<optimised out>,
stack_end=0x7fffffffde48) at ../csu/libc-start.c:310
#14 0x0000555555620a2a in _start ()


I have tried using PyImport_ImportModule or PyRun_SimpleString("import python_module") with the same effects... Any ideas?



I am running this on an ubuntu machine using python2.7.










share|improve this question













I am calling python code inside a C function and it works fine. However, I wanted to call the C function from python (effectively reuse all the C code) but when it comes to importing the python modules from C I get a seg. fault



(Removed all error checking from the code to make this simpler)



main.c:



#include "Python.h"

int runpython() {
Py_Initialize();
PyObject* pFunc;
PyObject *pArgs;
PyObject *pValue;
PyObject* pName = PyString_FromString("python_module");
PyObject* pModule = PyImport_Import(pName);
Py_DECREF(pModule);

pFunc = PyObject_GetAttrString(pModule, "pcall");
pArgs = PyTuple_New(1);
pValue = PyInt_FromLong(42);
PyTuple_SetItem(pArgs, 0, pValue);
pValue = PyObject_CallObject(pFunc, pArgs);
Py_DECREF(pArgs);
Py_DECREF(pValue);
Py_XDECREF(pFunc);
Py_DECREF(pModule);
}

int main() {
return runpython();
}


python_module.py



def pcall(value):
print "Value: " + str(value)


python_main.py



from ctypes import *

cdll.LoadLibrary("libMyPython.so")
libc = CDLL("libMyPython.so")
libc.runpython()


environment and execution:



export PYTHONPATH=$PYTHONPATH:$PWD
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$PWD

echo "Running C main"
gcc main.c -o main -I/usr/include/python2.7 -lpython2.7 -g
./main

echo "Running Python main"
gcc -shared -fPIC main.c -o libMyPython.so -I/usr/include/python2.7 -lpython2.7 -g
python python_main.py


A sample output is:



### Running C main
Value: 42
### Running Python main
Segmentation fault (core dumped)


Running in gdb yields the stack:



Program received signal SIGSEGV, Segmentation fault.
0x00005555556318d9 in PyImport_Import ()
(gdb) bt
#0 0x00005555556318d9 in PyImport_Import ()
#1 0x00007ffff5e07883 in runpython () at c_main.c:9
#2 0x00007ffff600edae in ffi_call_unix64 () from /usr/lib/x86_64-linux-gnu/libffi.so.6
#3 0x00007ffff600e71f in ffi_call () from /usr/lib/x86_64-linux-gnu/libffi.so.6
#4 0x00007ffff6221e56 in _ctypes_callproc ()
from /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so
#5 0x00007ffff6221505 in ?? () from /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so
#6 0x000055555564f420 in PyEval_EvalFrameEx ()
#7 0x0000555555647d0a in PyEval_EvalCodeEx ()
#8 0x0000555555647629 in PyEval_EvalCode ()
#9 0x000055555567861f in ?? ()
#10 0x0000555555673322 in PyRun_FileExFlags ()
#11 0x000055555567267d in PyRun_SimpleFileExFlags ()
#12 0x00005555556211ab in Py_Main ()
#13 0x00007ffff7a05b97 in __libc_start_main (main=0x555555620b10 <main>, argc=2,
argv=0x7fffffffde58, init=<optimised out>, fini=<optimised out>, rtld_fini=<optimised out>,
stack_end=0x7fffffffde48) at ../csu/libc-start.c:310
#14 0x0000555555620a2a in _start ()


I have tried using PyImport_ImportModule or PyRun_SimpleString("import python_module") with the same effects... Any ideas?



I am running this on an ubuntu machine using python2.7.







python c python-2.7 python-c-api






share|improve this question













share|improve this question











share|improve this question




share|improve this question










asked Nov 22 at 17:42









Evangelos Bempelis

234




234








  • 1




    Possible duplicate of How to clear a PyList object that contains objects?
    – DavidW
    Nov 23 at 7:15






  • 1




    I know the code is different but it's the same issue - you can't be calling Python code from a CDLL ctypes object because it has released the GIL.
    – DavidW
    Nov 23 at 7:16






  • 1




    (Alternatively this answer is also the same issue, but goes into a little more detail)
    – DavidW
    Nov 23 at 7:18










  • The last answer is straight spot on thank you. I would accept it as an answer if you submit it (or should this be marked as duplicate? I can see this being useful for otherts getting the seg fault)
    – Evangelos Bempelis
    Nov 23 at 10:04










  • I don't think I can mark it as duplicate (I found it after I'd already tried to mark the first answer, and I think that stops me from suggesting another for a couple of weeks) but hopefully someone else can
    – DavidW
    Nov 23 at 10:38














  • 1




    Possible duplicate of How to clear a PyList object that contains objects?
    – DavidW
    Nov 23 at 7:15






  • 1




    I know the code is different but it's the same issue - you can't be calling Python code from a CDLL ctypes object because it has released the GIL.
    – DavidW
    Nov 23 at 7:16






  • 1




    (Alternatively this answer is also the same issue, but goes into a little more detail)
    – DavidW
    Nov 23 at 7:18










  • The last answer is straight spot on thank you. I would accept it as an answer if you submit it (or should this be marked as duplicate? I can see this being useful for otherts getting the seg fault)
    – Evangelos Bempelis
    Nov 23 at 10:04










  • I don't think I can mark it as duplicate (I found it after I'd already tried to mark the first answer, and I think that stops me from suggesting another for a couple of weeks) but hopefully someone else can
    – DavidW
    Nov 23 at 10:38








1




1




Possible duplicate of How to clear a PyList object that contains objects?
– DavidW
Nov 23 at 7:15




Possible duplicate of How to clear a PyList object that contains objects?
– DavidW
Nov 23 at 7:15




1




1




I know the code is different but it's the same issue - you can't be calling Python code from a CDLL ctypes object because it has released the GIL.
– DavidW
Nov 23 at 7:16




I know the code is different but it's the same issue - you can't be calling Python code from a CDLL ctypes object because it has released the GIL.
– DavidW
Nov 23 at 7:16




1




1




(Alternatively this answer is also the same issue, but goes into a little more detail)
– DavidW
Nov 23 at 7:18




(Alternatively this answer is also the same issue, but goes into a little more detail)
– DavidW
Nov 23 at 7:18












The last answer is straight spot on thank you. I would accept it as an answer if you submit it (or should this be marked as duplicate? I can see this being useful for otherts getting the seg fault)
– Evangelos Bempelis
Nov 23 at 10:04




The last answer is straight spot on thank you. I would accept it as an answer if you submit it (or should this be marked as duplicate? I can see this being useful for otherts getting the seg fault)
– Evangelos Bempelis
Nov 23 at 10:04












I don't think I can mark it as duplicate (I found it after I'd already tried to mark the first answer, and I think that stops me from suggesting another for a couple of weeks) but hopefully someone else can
– DavidW
Nov 23 at 10:38




I don't think I can mark it as duplicate (I found it after I'd already tried to mark the first answer, and I think that stops me from suggesting another for a couple of weeks) but hopefully someone else can
– DavidW
Nov 23 at 10:38

















active

oldest

votes











Your Answer






StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});














draft saved

draft discarded


















StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53436016%2fpyimport-import-seg-faults-after-calling-c-that-calls-python-from-python%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown






























active

oldest

votes













active

oldest

votes









active

oldest

votes






active

oldest

votes
















draft saved

draft discarded




















































Thanks for contributing an answer to Stack Overflow!


  • Please be sure to answer the question. Provide details and share your research!

But avoid



  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.


To learn more, see our tips on writing great answers.





Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


Please pay close attention to the following guidance:


  • Please be sure to answer the question. Provide details and share your research!

But avoid



  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.


To learn more, see our tips on writing great answers.




draft saved


draft discarded














StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53436016%2fpyimport-import-seg-faults-after-calling-c-that-calls-python-from-python%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







Popular posts from this blog

A CLEAN and SIMPLE way to add appendices to Table of Contents and bookmarks

Calculate evaluation metrics using cross_val_predict sklearn

Insert data from modal to MySQL (multiple modal on website)