Making C++ Dynamic with SWIG

Sometimes it seems like I’ve spent half of my programming life hooking up dynamic scripting front-ends to C/C++ back-end code. I did it when I worked on PsyScope and E-Prime, and for my own research code in Guile, PLT Scheme, and now Python. It’s amazing how similar and regular the process is in all the different languages, and how tedious it is to manually register each C/C++ entity that has to be visible to the scripting languages. Luckily, computers are great at automating regular, tedious procedures: enter SWIG the Simple Wrapper and Interface Generator. SWIG works with many languages including Python, PERL, Guile, PLT Scheme, Ruby, PHP, and even Java.

SWIG so simplifies the task that I never write C++ programs with main() functions anymore. I just write libraries that get wrapped by SWIG and loaded into Python as modules, allowing me to use all of Python as my user interface. It also makes it simple to integrate someone else’s C/C++ code into my existing Python codebase. I used it to make a version of the Stage robot simulator that can be loaded into Python as a module, instead of running as a separate process.

My latest SWIG project was much simpler. I wanted to use Rich Sutton’s Tile Coding software for converting continuous features (like LIDAR returns) into discrete boolean features. Unlike the work with Stage, this only involved wrapping an interface to one C function and one constant, and it was basically done in 30 minutes (including downloading the library code from Rich’s site, etc). Still, I felt like I learned something useful about SWIG.

The function I wrapped [GetTiles()] has lots of parameters, 4 of which are an input array and its length and an output array to be filled, and its length. Naturally, in Python, you’d like for the function to simply take the input array, sans length, and return the output array as a return value, rather than a parameter. I was a little worried when I started that it would be hard to figure out how to tell SWIG to generate the appropriate C code to convert all the arguments properly. However, when SWIG generates a wrapper, it creates code on both sides of the Python/C boundary. It creates “wrapper” functions in C that convert the arguments to and from the Python interpreter’s internal format. SWIG provides extremely flexible “typemaps” for specifying the details of these typemaps, but its not always obvious how to write a typemap to do some specific thing. Luckily, for Python, code SWIG also generates “proxy” code in Python, that further wraps (some of the) calls to the C/C++ wrappers. Furthermore, it allows the programmer to override the Python proxy for some wrapped function, letting me write the argument conversion for GetTiles() in Python, where it’s easy, rather than in C++, where I’m still not sure exactly how I would do it. By the same process, I can add auxiliary functions directly to the new wrapped module, written entirely in Python, and specified in my interface file.

With all the great dynamic languages out there, and tools like SWIG for grafting together dynamic languages and C++, I don’t understand why people would want to write entire programs in C++ (or, god forbid, C) anymore, unless they have specific resource limitations that require it like O.S. device drivers and embedded programs.


I have not failed…

I have not failed. I’ve just found 10,000 ways that won’t work.
— Thomas Edison
(seen on Now This log)

This statement of perseverance seems particularly apropos of life as a PhD student, and it makes me feel good. Sometimes you have to find 10,000 ways that won’t work so that you can find the 10,001st way that does work.

I haven’t been posting lately because I’ve been super busy. In the last few weeks I finished up the camera-ready copy for a paper I have in the AAAI-2004 Workshop on Learning and Planning in Markov Processes and then submitted a revised paper on the same topic to NIPS04. I feel like all the slogging along getting my code infrastructure together and setting things up so I can run batches of simulations in parallel on our new cluster is paying off, and I’m finally to the point where I can do science, and start thinking about the actual scientific questions I want to investigate, rather than how to get my simulations to run fast enough that I can get results back while I’m still (relatively) young.

I’ll save the details of the two papers for a different posting, though you can find the workshop paper on my research page.