passing python strings, through cython, to C

Question

I am trying to write a module with some c and some python parts. I am using cython to bridge the gap.

I want to store my (very long) string constants in python, because of the much nicer syntax:

const char long_string = "\npart of string\n" "next part\n" "last part\n";

versus:

long_string = """ part of string next part last part """

(the strings are much longer than this, and more complicated - to the extent that I don't want to have to add and remove the "s and \n"s every time I want to edit it with syntax highlighting. In fact, they are openCL kernels.)

I need to be able to turn these into c strings using cython, and according to the documentation I should just need this:

cdef bytes py_bytes = py_string.encode() cdef char* c_string = py_bytes

and with no manual memory management, c_string will work as long as I keep a reference to py_bytes.

However, I can't get this working with a simple printf test. Here is my cython file:

cdef extern from "stdio.h": printf(char* string) def go(): py_string = """ a complicated string with a few newlines. """ cdef bytes py_bytes = py_string.encode() cdef char* c_string = py_bytes printf(c_string) print "we don't get this far :("

which, when compiled at runtime using pyximport gives the following output to terminal before segfaulting:

a complicated string with a few newlines. Segmentation fault: 11

now, I've checked what cython actually puts in the c file, and tried it in a vanilla C file where it doesn't segfault:

#include "stdio.h" static char __pyx_k_1[] = "\na complicated string\nwith a few\nnewlines.\n"; int main(void) { void* output = printf(__pyx_k_1); if (!output) { printf("apparently, !output."); } }

to be clear, cython generates code which catches the output of printf and tests for "not that". the type of the variable is a PyObject*.

My only guess here was that the string was improperly terminated, so printf just carries on past the end of it and causes the segfault, but since that doesn't happen in my pure c test, I'm completely stumped.

So, my actual question is how do I really pass a c-string to c code from cython? Answers pointing out an easier way to solve the actual problem I'm trying to solve at the top are also very welcome :)

Fred Foo · Accepted Answer · 2011-11-21 17:27:29Z

Importing printf from libc.stdio fixes the problem for me:

from libc.stdio cimport printf def go(): py_string = """ a complicated string with a few newlines. """ cdef bytes py_bytes = py_string.encode() cdef char* c_string = py_bytes printf(c_string) print "we actually got this far! :)"

The error is in the declaration of printf. That should have been, as stdio.pxd lists,

cdef extern from *: ctypedef char const_char "const char" int printf(const_char *, ...)

whereas your version is implicitly object printf(char *); the default return value type is Python object rather than int as in C. Getting the right declaration turns off Cython's attempt to Py_XDECREF the return value from printf.

(Btw, in your "vanilla" C problem, you shouldn't be casting the return value from printf to void *.)

Thank you, this solves the problem immediately. I had clearly misunderstood cimport - I thought you had to specify the types exactly.
@tehwalrus: you didn't specify the type exactly. It should have been int printf(const_char *, ...) where you have to ctypedef char const_char "const char" manually. I think Cython presumes a return type of object rather than int.
@tehwalrus: in fact, I checked the generated code again and this is exactly what went wrong. See updated answer for an explanation of the error.
Many thanks for the detailed answer. I will be more careful in my definitions in future! :)

Collectives™ on Stack Overflow

passing python strings, through cython, to C

1 Answer 1

4 Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Linked

Related