If you know C, adding new Python built-in modules is easy. There are two things you can’t do directly with Python that you can do with Extension Modules: implement new built-in object types; C library functions and system calls are called.

To support extensions, the Python API (Application Programming Interface) defines a series of functions, macros, and variables that provide access to most of the Python runtime system. Python’s API can be used by referencing the “python.h” header in a C source file.

How extensions are written depends on your goals and system setup; More on this in the following sections.

Note: The C extension interface specifically refers to CPython, and extension modules do not work on other Python implementations. In most cases, you should avoid writing C extensions to preserve portability. For example, if your use case calls the C library or system call, you should consider using the Ctypes module or the CFFI library rather than writing YOUR own C code. These modules allow you to write Python code to interface with C code and are much more portable. Somehow the compilation failed.

1. A simple example

For example, we have a C function like this:

int great_function(int a) {
    return a + 1;
}
Copy the code

Expect Python to use this:

>>> from great_module import great_function 
>>> great_function(2)
3
Copy the code

Let’s take the simplest case. We put powerful functions in the C file great_module.c.

#include <Python.h> int great_function(int a) { return a + 1; } static PyObject * _great_function(PyObject *self, PyObject *args) { int _a; int res; if (! PyArg_ParseTuple(args, "i", &_a)) return NULL; res = great_function(_a); return PyLong_FromLong(res); } static PyMethodDef GreateModuleMethods[] = { { "great_function", _great_function, METH_VARARGS, "" }, {NULL, NULL, 0, NULL} }; PyMODINIT_FUNC initgreat_module(void) { (void) Py_InitModule("great_module", GreateModuleMethods); }Copy the code

In addition to the function great_function, there are the following sections in this file:

  • Wrap function _great_function. It is responsible for converting Python arguments to C arguments (PyArg_ParseTuple), calling the actual great_function, and processing the return value of great_Function, which is eventually returned to the Python environment.
  • Export table GreateModuleMethods. It is responsible for telling Python which functions in this module can be called by Python. The name of the exported table can be arbitrary, and each entry takes four arguments: the first argument is the name of the function provided to the Python environment, and the second argument is _great_function, the wrapped function. The third argument is a longer argument, and the fourth argument is an illustrative string. Exported tables always end with {NULL, NULL, 0, NULL}.
  • Export the function initgreat_module. The name of this module is not arbitrary. Add init to your module name. The export function concatenates the module name with the export table.

2. Header files

In the code we import such a header file

#include <Python.h>
Copy the code

This imports the Python API (you can add comments here describing the module’s goals and copyright information if you prefer).

Note: Since Python may define preprocessor definitions that affect standard header files on some systems, you must include python.h before including any standard header files. It is recommended that PY_SSIZE_T_CLEAN be always defined before python.h. See the arguments to the extract extension function to learn more about this macro.

Except for those already defined in header files, all user-visible symbols are defined in python.h and have the prefix Py or Py. For convenience, and to make the Python interpreter widely available, “python.h” also includes a few standard header files:

,

,

, and

. If the following header file does not exist on your system, the functions malloc(), free(), and realloc() are declared directly.



3. Compile and use

Under Windows, the command to compile this file at the Visual Studio command prompt is

cl /LD great_module.c /o great_module.pyd -IC:\Python27\include C:\Python27\libs\python27.lib
Copy the code

/LD is to generate dynamic link library. After successful compilation, you can get great_module.pyd (actually DLL) in the current directory. This PYD can be used directly as a Module in the Python environment.

1.4 Under Linux, use GCC to compile:

GCC -fpic-shared great_module.c -o great_module.so -i /usr/include/python2.7/-lpython2.7Copy the code

Get great_module.so in the current directory, which can be used directly in Python.

4. Wrapping function

Let’s look at the wrap function _great_function:

static PyObject * _great_function(PyObject *self, PyObject *args) { int _a; int res; if (! PyArg_ParseTuple(args, "i", &_a)) return NULL; res = great_function(_a); return PyLong_FromLong(res); }Copy the code

There is a way to convert python argument lists directly to C functions. C functions always take two arguments, usually named self and args.

For module-level functions, the self argument points to the module object; For object instances, it points to methods.

The args argument is a pointer to a Python tuple object that contains arguments. Each tuple item corresponds to a call parameter. These arguments are also all Python objects – to use them in our C functions we need to convert them to C values first. The function PyArg_ParseTuple() in the Python API checks the parameter type and converts it to a C value. It uses the template string to determine the type of parameter needed and the type of the C variable that stores the converted value. Details will be explained later.

PyArg_ParseTuple() normally returns true(non-zero) and has stored each variable value at the address provided. If there is an error (zero), the function should return NULL to inform the interpreter of the error (as seen in the example).

Take a look at the following code:

if (! PyArg_ParseTuple(args, "s", &command)) return NULL;Copy the code

If an error is detected in the argument list, NULL (the error indicator of the function that returns an object pointer) is returned, according to the exception set by PyArg_ParseTuple(). In other cases the string value of the parameter is copied to the local variable command. This is a pointer assignment, and you should not modify the string to which it points (so in standard C, the variable command should be properly declared as const char *command).

The next statement uses the function great_function() and passes it the argument just fetched from PyArg_ParseTuple() :

res = great_function(_a);
Copy the code

Our res = great_function() function must return the value of res as a Python object. This is done using the function PyLong_FromLong().

return PyLong_FromLong(res);
Copy the code

In this case, an integer object is returned (which is managed in the Python heap).

If your C function has no useful return value (a function that returns void), you must return None. (You can do this with the Py_RETUN_NONE macro):

Py_INCREF(Py_None);
return Py_None;
Copy the code

Py_None is a C name specifying the Python object None. This is a real PY object, not a NULL pointer.

5. Module method table

To show how great_function() is called by a Python program. To declare a function to be called by Python, you need to define a “method table”.

static PyMethodDef GreateModuleMethods[] = {
    {
        "great_function",
        _great_function,
        METH_VARARGS,
        ""
    },
    {NULL, NULL, 0, NULL}
};
Copy the code

Notice the third parameter (METH_VARARGS), which specifies the calling convention that will use C. Optional values are METH_VARARGS, METH_VARARGS | METH_KEYWORDS. The value 0 represents stale variables using PyArg_ParseTuple().

If METH_VARARGS is used alone, the function waits for Python to pass in arguments in the form of a tuple and is eventually parsed using PyArg_ParseTuple().

METH_KEYWORDS indicates that keyword parameters are accepted. In this case, the C function needs to accept a third PyObject * object representing the dictionary argument, which is resolved using PyArg_ParseTupleAndKeywords().

6. Initialize the function

PyMODINIT_FUNC initgreat_module(void) {
    (void) Py_InitModule("great_module", GreateModuleMethods);
}
Copy the code