Pure Python Mode

The Cython compiler supports two main syntax styles. While we present the traditional .pyx syntax for comparison, this page primarily focuses on teaching the modern Pure Python syntax (used in .py files), as it allows for easier integration with existing Python codebases.

Note

This page uses two different syntax variants:

• Cython specific cdef syntax, which was designed to make type declarations concise and easily readable from a C/C++ perspective.

• Pure Python syntax which allows static Cython type declarations in pure Python code, following PEP-484 type hints and PEP 526 variable annotations.

  To make use of C data types in Python syntax, you need to import the special cython module in the Python module that you want to compile, e.g.

  import cython 

  If you use the pure Python syntax we strongly recommend you use a recent Cython 3 release, since significant improvements have been made here compared to the 0.29.x releases.

In some cases, it’s desirable to speed up Python code without losing the ability to run it with the Python interpreter. While pure Python scripts can be compiled with Cython, it usually results only in a speed gain of about 20%-50%.

To go beyond that, Cython provides language constructs to add static typing and cythonic functionalities to a Python module to make it run much faster when compiled, while still allowing it to be interpreted. This is accomplished via an augmenting .pxd file, via Python type PEP-484 type annotations (following PEP 484 and PEP 526), and/or via special functions and decorators available after importing the magic cython module. All three ways can be combined at need, although projects would commonly decide on a specific way to keep the static type information easy to manage.

Although it is not typically recommended over writing straight Cython code in a .pyx file, there are legitimate reasons to do this - easier testing and debugging, collaboration with pure Python developers, etc. In pure mode, you are more or less restricted to code that can be expressed (or at least emulated) in Python, plus static type declarations. Anything beyond that can only be done in .pyx files with extended language syntax, because it depends on features of the Cython compiler.

Augmenting .pxd

Using an augmenting .pxd allows to leave the original .py file completely untouched. On the other hand, one needs to maintain both the .pxd and the .py to keep them in sync.

While declarations in a .pyx file must correspond exactly with those of a .pxd file with the same name (and any contradiction results in a compile time error, see pxd files), the untyped definitions in a .py file can be overridden and augmented with static types by the more specific ones present in a .pxd.

If a .pxd file is found with the same name as the .py file being compiled, the compiler will convert the corresponding definitions in the .py file to be of the declared type. The following tabs illustrate the process, showing the original Python file and the final equivalent Cython code, which are both defined by the declarations in the supplementing .pxd file:

Pure PythonCython

A.py

def myfunction(x, y=2): a = x - y return a + x * y def _helper(a): return a + 1 class A: def __init__(self, b=0): self.a = 3 self.b = b def foo(self, x): print(x + _helper(1.0)) 

The declarations required to enable type conversion for the pure Python example are defined in the supplementing A.pxd file:

A.pxd

cpdef int myfunction(int x, int y=*) cdef double _helper(double a) cdef class A: cdef public int a, b cpdef foo(self, double x) 

Notice how in order to provide the Python wrappers to the definitions in the .pxd, that is, to be accessible from Python,

• Python visible function signatures must be declared as cpdef (with default arguments replaced by a * to avoid repetition):

  cpdef int myfunction(int x, int y=*) 

• C function signatures of internal functions can be declared as cdef:

  cdef double _helper(double a) 

• cdef classes (extension types) are declared as cdef class;

• cdef class attributes must be declared as cdef public if read/write Python access is needed, cdef readonly for read-only Python access, or plain cdef for internal C level attributes;

• cdef class methods must be declared as cpdef for Python visible methods or cdef for internal C methods.

In the example above, the type of the local variable a in myfunction() is not fixed and will thus be a Python object. To statically type it, one can use Cython’s @cython.locals decorator (see Magic Attributes, and Magic Attributes within the .pxd).

Normal Python (def) functions cannot be declared in .pxd files. It is therefore currently impossible to override the types of plain Python functions in .pxd files, e.g. to override types of their local variables. In most cases, declaring them as cpdef will work as expected.

Magic Attributes

Special decorators are available from the magic cython module that can be used to add static typing within the Python file, while being ignored by the interpreter.

This option adds the cython module dependency to the original code, but does not require to maintain a supplementary .pxd file. Cython provides a fake version of this module as Cython.Shadow, which is available as cython.py when Cython is installed, but can be copied to be used by other modules when Cython is not installed.

“Compiled” switch

• compiled is a special variable which is set to True when the compiler runs, and False in the interpreter. Thus, the code

  if cython.compiled: print("Yep, I'm compiled.") else: print("Just a lowly interpreted script.") 

  will behave differently depending on whether or not the code is executed as a compiled extension (.so/.pyd) module or a plain .py file.

Static typing

• cython.declare declares a typed variable in the current scope, which can be used in place of the cdef type var [= value] construct. This has two forms, the first as an assignment (useful as it creates a declaration in interpreted mode as well):

  Pure PythonCython

  cython_declare.py

  x = cython.declare(cython.int) y = cython.declare(cython.double, 0.57721) 

  and the second mode as a simple function call:

  Pure PythonCython

  cython_declare2.py

  # Declare C-typed variables in Pure Python mode cython.declare(x=cython.int, y=cython.double) 

  It can also be used to define extension type private, readonly and public attributes:

  Pure PythonCython

  cclass.py

  # This is the Pure Python mode version, using cython.declare and annotations. @cython.cclass class A: cython.declare(a=cython.int, b=cython.int) c = cython.declare(cython.int, visibility='public') d = cython.declare(cython.int) # private by default. e = cython.declare(cython.int, visibility='readonly') def __init__(self, a, b, c, d=5, e=3): self.a = a self.b = b self.c = c self.d = d self.e = e 

• @cython.locals is a decorator that is used to specify the types of local variables in the function body (including the arguments):

  Pure PythonCython

  locals.py

  @cython.locals(a=cython.long, b=cython.long, n=cython.longlong) def foo(a, b, x, y): n = a * b # ... 

• @cython.returns(<type>) specifies the function’s return type.

• @cython.exceptval(value=None, *, check=False) specifies the function’s exception return value and exception check semantics as follows:

  Pure PythonCython

  exceptval_syntax.py

  from cython import exceptval, cfunc, int as cython_int @exceptval(-1) @cfunc def func_a() -> cython_int: return 0 @exceptval(-1, check=False) @cfunc def func_b() -> cython_int: return 0 @exceptval(check=True) @cfunc def func_c() -> cython_int: return 0 @exceptval(-1, check=True) @cfunc def func_d() -> cython_int: return 0 @exceptval(check=False) @cfunc def func_e() -> cython_int: return 0 

  If exception propagation is disabled, any Python exceptions that are raised inside of the function will be printed and ignored.

C types

There are numerous types built into the Cython module. It provides all the standard C types, namely char, short, int, long, longlong as well as their unsigned versions uchar, ushort, uint, ulong, ulonglong. The special bint type is used for C boolean values and Py_ssize_t for (signed) sizes of Python containers.

For each type, there are pointer types p_int, pp_int, etc., up to three levels deep in interpreted mode, and infinitely deep in compiled mode. Further pointer types can be constructed with cython.pointer[cython.int] (or cython.pointer(cython.int) for compatibility with Cython versions before 3.1), and arrays as cython.int[10]. A limited attempt is made to emulate these more complex types when running in a Python interpreter, but only so much can be done from the Python language.

Pure python mode also supports type qualifiers const and volatile, which can be applied using cython.const[cython.int] and cython.volatile[cython.int]. For more details see Type qualifiers section.

The Python type annotations bool and float are aliases for C bint and C double respectively. The Python int type does not have an equivalent in C and is therefore not aliased to any C type but means the builtin Python type in annotations. Also, the Python builtin types list, dict, tuple, etc. may be used, as well as any user defined types.

Typed C-tuples can be declared as a a bare tuple of C types or as tuple[...] or typing.Tuple[...] with the respective item types listed in the brackets.

Extension types and cdef functions

• The class decorator @cython.cclass creates a cdef class.

• The function/method decorator @cython.cfunc creates a cdef function.

• @cython.ccall creates a cpdef function, i.e. one that Cython code can call at the C level.

• @cython.locals declares local variables (see above). It can also be used to declare types for arguments, i.e. the local variables that are used in the signature.

• @cython.inline is the equivalent of the C inline modifier.

• @cython.final terminates the inheritance chain by preventing a type from being used as a base class, or a method from being overridden in subtypes. This enables certain optimisations such as inlined method calls.

Here is an example of a cdef function:

Pure PythonCython

c_compare.py

@cython.cfunc def c_compare(a: cython.int, b: cython.int) -> cython.bint: return a == b 

Managing the Global Interpreter Lock

• cython.nogil can be used as a context manager or as a decorator to replace the nogil keyword:

  Pure PythonCython

  nogil_example.py

  with cython.nogil: pass # Placeholder for code intended to run without the GIL @cython.nogil @cython.cfunc @cython.returns(cython.int) def func_not_needing_the_gil() -> cython.int: return 1 

  Note that the two uses differ: the context manager releases the GIL while the decorator marks that a function can be run without the GIL. See Cython and the GIL for more details.

• cython.gil can be used as a context manager to replace the gil keyword:

  with cython.gil: # code block with the GIL acquired 

  Note

  Cython currently does not support the @cython.with_gil decorator.

Both directives accept an optional boolean parameter for conditionally releasing or acquiring the GIL. The condition must be constant (at compile time):

with cython.nogil(False): # code block with the GIL not released @cython.nogil(True) @cython.cfunc def func_released_gil() -> cython.int: # function with the GIL released with cython.gil(False): # code block with the GIL not acquired with cython.gil(True): # code block with the GIL acquired 

A common use case for conditionally acquiring and releasing the GIL are fused types that allow different GIL handling depending on the specific type (see Conditional Acquiring / Releasing the GIL).

cimports

The special cython.cimports package name gives access to cimports in code that uses Python syntax. Note that this does not mean that C libraries become available to Python code. It only means that you can tell Cython what cimports you want to use, without requiring special syntax. Running such code in plain Python will fail.

Pure PythonCython

py_cimport.py

from cython.cimports.libc import math def use_libc_math(): return math.ceil(5.5) 

Since such code must necessarily refer to the non-existing cython.cimports ‘package’, the plain cimport form cimport cython.cimports... is not available. You must use the form from cython.cimports....

Further Cython functions and declarations

• address is used in place of the & operator:

  Pure PythonCython

  address.py

  cython.declare(x=cython.int, x_ptr=cython.p_int) x_ptr = cython.address(x) 

• sizeof emulates the sizeof operator. It can take both types and expressions.

  Pure PythonCython

  sizeof.py

  cython.declare(n=cython.longlong) print(cython.sizeof(cython.longlong)) print(cython.sizeof(n)) 

• typeof returns a string representation of the argument’s type for debugging purposes. It can take expressions.

  Pure PythonCython

  typeof.py

  import cython cython.declare(n=cython.longlong) n = 0 print(type(n)) 

• struct can be used to create struct types:

  Pure PythonCython

  struct.py

  MyStruct = cython.struct( x=cython.int, y=cython.int, data=cython.double) a: MyStruct 

• union creates union types with exactly the same syntax as struct.

• typedef defines a type under a given name:

  Pure PythonCython

  typedef.py

  T = cython.typedef(cython.p_int) cython.declare(my_ptr=T) 

• cast will (unsafely) reinterpret an expression type. cython.cast(T, t) is equivalent to <T>t. The first attribute must be a type, the second is the expression to cast. Specifying the optional keyword argument typecheck=True has the semantics of <T?>t:

  Pure PythonCython

  cast.py

  obj = [1, 2, 3] l1 = cython.cast(list, obj) l2 = cython.cast(list, obj, typecheck=True) 

• fused_type creates a new type definition that refers to the multiple types. The following example declares a new type called my_fused_type which can be either an int or a double:

  Pure PythonCython

  fused_type.py

  my_fused_type = cython.fused_type( cython.int, cython.float) 

Magic Attributes within the .pxd

The special cython module can also be imported and used within the augmenting .pxd file. For example, the following Python file dostuff.py:

def dostuff(n): t = 0 for i in range(n): t += i return t 

can be augmented with the following .pxd file dostuff.pxd:

import cython @cython.locals(t=cython.int, i=cython.int) cpdef int dostuff(int n) 

The cython.declare() function can be used to specify types for global variables in the augmenting .pxd file.

PEP-484 type annotations

Python type hints can be used to declare argument types, as shown in the following example:

Pure PythonCython

annotations.py

@cython.ccall def func(foo: dict, bar: cython.int) -> tuple: foo["hello world"] = 3 + bar return foo, 5 

Note the use of cython.int rather than int - Cython does not translate an int annotation to a C integer by default since the behaviour can be quite different with respect to overflow and division.

Annotations on global variables are currently ignored. This is because we expect annotation-typed code to be in majority written for Python, and global type annotations would turn the Python variable into an internal C variable, thus removing it from the module dict. To declare global variables as typed C variables, use @cython.declare().

Annotations can be combined with the @cython.exceptval() decorator for non-Python return types:

Pure PythonCython

exceptval.py

@cython.exceptval(-1) @cython.cfunc def func(x: cython.int) -> cython.int: if x < 0: raise ValueError("need integer >= 0") return x + 1 

Note that the default exception handling behaviour when returning C numeric types is to check for -1, and if that was returned, check Python’s error indicator for an exception. This means, if no @exceptval decorator is provided, and the return type is a numeric type, then the default with type annotations is @exceptval(-1, check=True), in order to make sure that exceptions are correctly and efficiently reported to the caller. Exception propagation can be disabled explicitly with @exceptval(check=False), in which case any Python exceptions raised inside of the function will be printed and ignored.

Since version 0.27, Cython also supports the variable annotations defined in PEP 526. This allows to declare types of variables in a Python 3.6 compatible way as follows:

Pure PythonCython

pep_526.py

def func(): # Cython types are evaluated as for cdef declarations x: cython.int y: cython.double = 0.57721 z: cython.float = 0.57721 # Python types shadow Cython types for compatibility reasons a: float = 0.54321 b: int = 5 pass @cython.cclass class A: a: cython.int b: cython.int def __init__(self, b=0): self.a = 3 self.b = b 

There is currently no way to express the visibility of object attributes.

Disabling annotations

To avoid conflicts with other kinds of annotation usages, Cython’s use of annotations to specify types can be disabled with the annotation_typing compiler directive. From Cython 3 you can use this as a decorator or a with statement, as shown in the following example:

Pure PythonCython

disabled_annotations.py

@cython.annotation_typing(False) def function_without_typing(a: int, b: int) -> int: c: int = a + b return c * a @cython.annotation_typing(False) @cython.cclass class NotAnnotatedClass: d: dict def __init__(self, dictionary: dict): self.d = dictionary @cython.annotation_typing(True) def annotated_method(self, key: str, a: cython.int, b: cython.int): prefixed_key: str = 'prefix_' + key self.d[prefixed_key] = a + b def annotated_function(a: cython.int, b: cython.int): s: cython.int = a + b with cython.annotation_typing(False): # Cython is ignoring annotations within this code block c: list = [] c.append(a) c.append(b) c.append(s) return c 

typing Module

Support for the full range of annotations described by PEP-484 is not yet complete. Cython 3 currently understands the following features from the typing module:

• Optional[tp], which is interpreted as tp or None;

• Union[tp, None] or Union[None, tp], which is interpreted as tp or None;

• typed containers such as List[str], which is interpreted as list. The hint that the elements are of type str is currently ignored;

• Tuple[...], which is converted into a Cython C-tuple where possible and a regular Python tuple otherwise.

• ClassVar[...], which is understood in the context of cdef class or @cython.cclass.

Some of the unsupported features are likely to remain unsupported since these type hints are not relevant for the compilation to efficient C code. In other cases, however, where the generated C code could benefit from these type hints but does not currently, help is welcome to improve the type analysis in Cython.

Reference table

The following reference table documents how type annotations are currently interpreted. Cython 0.29 behaviour is only shown where it differs from Cython 3.0 behaviour. The current limitations will likely be lifted at some point.

Annotation typing rules

Feature	Cython 0.29	Cython 3.0
int	Any Python object	Exact Python int (language_level=3 only)
float		C double
Builtin type e.g. dict, list		Exact type (no subclasses), not None
Extension type defined in Cython		Specified type or a subclasses, not None
cython.int, cython.long, etc.		Equivalent C numeric type
typing.Optional[any_type]	Not supported	Specified type (which must be a Python object), allows None
typing.Union[any_type, None]	Not supported	Specified type (which must be a Python object), allows None
any_type | None	Not supported	Specified type (which must be a Python object), allows None
typing.List[any_type] (and similar)	Not supported	Exact list, with the element type ignored currently
typing.ClassVar[...]	Not supported	Python-object class variable (when used in a class definition)

Tips and Tricks

Avoiding the cython runtime dependency

Python modules that are intended to run both compiled and as plain Python usually have the line ìmport cython in them and make use of the magic attributes in that module. If not compiled, this creates a runtime dependency on Cython’s shadow module that provides fake implementations of types and decorators.

Code that does not want to require Cython or its shadow module as as runtime dependency at all can often get away with a simple, stripped-down replacement like the following:

try: import cython except ImportError: class _fake_cython: compiled = False def cfunc(self, func): return func def ccall(self, func): return func def __getattr__(self, type_name): return "object" cython = _fake_cython() 

Calling C functions

The magic cython.cimports package provides a way to cimport external compile time C declarations from code written in plain Python. For convenience, it also provides a fallback Python implementation for the libc.math module.

However, it is normally not possible to call C functions in pure Python code as there is no general way to represent them in normal (uncompiled) Python. But in cases where an equivalent Python function exists, this can be achieved by combining C function coercion with a conditional import as follows:

# mymodule.pxd # declare a C function as "cpdef" to export it to the module cdef extern from "math.h": cpdef double sin(double x) 

# mymodule.py import cython # override with Python import if not in compiled code if not cython.compiled: from math import sin # calls sin() from math.h when compiled with Cython and math.sin() in Python print(sin(0)) 

Note that the “sin” function will show up in the module namespace of “mymodule” here (i.e. there will be a mymodule.sin() function). You can mark it as an internal name according to Python conventions by renaming it to “_sin” in the .pxd file as follows:

cdef extern from "math.h": cpdef double _sin "sin" (double x) 

Pure PythonCython

internal_sin.py

if not cython.compiled: from math import sin as _sin print(_sin(0)) 

You would then also change the Python import to from math import sin as _sin to make the names match again.

Using C arrays for fixed size lists

C arrays can automatically coerce to Python lists or tuples. This can be exploited to replace fixed size Python lists in Python code by C arrays when compiled. An example:

Pure PythonCython

c_arrays.py

import cython @cython.locals(counts=cython.int[10], digit=cython.int) def count_digits(digits):  """  >>> digits = '01112222333334445667788899'  >>> count_digits(map(int, digits))  [1, 3, 4, 5, 3, 1, 2, 2, 3, 2]  """ counts = [0] * 10 for digit in digits: assert 0 <= digit <= 9 counts[digit] += 1 return counts 

In normal Python, this will use a Python list to collect the counts, whereas Cython will generate C code that uses a C array of C ints.