"ctypes" made easy

`ctypes` made easy

Peter Edwards, Arista Networks.
peadar@arista.com

ctypes, the basics

What's ctypes,then?

FFI for python - call C functions from python

How does this differ from a python module?

Functions don't have to be python-aware - they take standard C types as arguments

How's it work?

ctypes library provides basic types
applications can assemble descriptions compound types like structures, arrays, and pointer types
ctypes uses this data, and it's understanding of the calling conventions of your platform to construct the stack frames and register context that C functions expect to see on entry

What use is it?

We use it for writing tests
A test failure can leave you in a REPL where you can poke at the internal details of a process

ctypes, the basics - an example

output:

Ugh. Boilerplate

Repeats what's in the C source, with different syntax.

Can we get the compiler to output something for us?

Actually, it already does

Here's where I talk about DWARF again

Output from

objdump
                        --dwarf=info libbasic.so >
                        basic-objdump.out

CTypeGen in action

We can use this:

To generate this:

Now, how does our boilerplate look?

old:

new:

As well as reducing the size of the code, changes in the C code are automatically updated in the generated python, so if we change the return value of our function, to, say, double, there are no nasty surprises.

As we use more types, the savings increase. We've removed thousands of lines of hand-written boilerplate.

Bonus feature: mocking C functions with Python

Sometimes we want to test a function, but want to mock out some of the things it calls.


                    extern void entry(int expect_return, int expect_i); // this is what we want to test...
                    extern int f(int ival, const char *sval, int * ipval); // ... but it calls this

Easy!


@CMock.Mock(lib.f, lib)
def mockedF(i, s, iptr):
   iptr[0] = 101
   return 100

Now, when the C function entry is called, instead of calling the f function from the C program, it calls our python function, mockedF

How it works

ctypes allows us to pass python "callbacks" to C functions, just like we can pass ints and doubles - it creates a C callable pointer-to-function from your python function.
```
 
                                     
```
```
 
                                     
```
For calling functions that have external linkage, the runtime linker maintains a table of offsets that allow it to lazily resolve function names to executable code
We can hijack this "Global Offset Table" to intercept any calls to functions that are resolved at runtime, and direct them to the code generated by ctypes above.
If we hijack the entry for "f" above with the raw code that ctypes generates in the above case, we can have that called in "f"'s stead.

Acknowledgements

Lots of people helped with this, with code, suggestions, testing, and helping make it open source.

Denis Evoy reviewed lots of my bad decisions, and made me fix them. He, Joanne Mikkellson, Sharad Birmiwal, Asang Dani, and Josh Pfosi all made lots of suggestions and found lots of bugs that made it better.

Pawel Kurdybacha helped me prepare the source for github.

Thanks to Arista for being the kind of place that lets me do cool things like this, and then share them with the rest of the world

ctypes made easy