Embedded PyObjC

When people think of a PyObjC application, they usually think of a Python application that uses Objective-C libraries. However, it's also possible to do the opposite: An Objective-C application that embeds Python code through a plugin. Building an application this way has advantages (speed, integration and memory usage) and should be used more often. This article explains why and how to achieve this.


Speed. In a GUI application, there's usually many, many calls being made all the time by the different elements of the GUI. For example, NSTableView's datasource and delegate methods are called tons of times at each redraw. Each calls having to pass through the PyObjC bridge is inherently slower than a native call. While machines are usually fast enough for this not to be noticeable most of the time, there might be situations where it's not the case, scrolling a large list for example.

I have no benchmarks to back this. All I can tell you is that my first PyObjC application, musicGuru, was initially all Python. Scrolling was sluggish and I decided to give the Embedded Way a try. It fixed the problem and I never looked back again.

Integration. Although XCode and Interface Builder offer Python integration (and since I never used them, I don't know if it works well), there's no doubt that Apple's efforts are much more axed on Objective-C. Auto-completion, help, build process, these are all designed with Objective-C in mind.

Memory usage. Since PyObjC 2.0, metadata from Apple's bridge support files are loaded in memory, leading to pretty high initial memory usage (as I mentioned in my article about 64-bit PyObjC applications). When you embed Python in your Objective-C application, it usually means that your Python code use fewer Objective-C classes, thus allowing you to use neat tricks to reduce that memory usage (more about this below). I'm talking about saving tens of MB of initial memory usage here, so it's not negligible.


Update 2010-11-23: I published a project containing a minimal application with embedded PyObjC. You might want to take a look.

The PyObjC website has an old tutorial about embedding Python [dead link] already, but it unfortunately doesn't explain much besides the basics. One thing it doesn't explain (because all instantiations in the example take place in the NIB) is how to locate a class in a plugin, interface it and instantiate it. This is what I'm going to do right away with an example.

Let's say that we want to build a PyObjC application embedding Python that simply displays a list of strings in a NSTableView. Let's first write our Python class:

class Foobar(object):
    def __init__(self):
        self.strings = ['foo', 'bar', 'baz']

    def count(self):
        return len(self.strings)

    def string_at_index(self, index):
        return self.strings[index]

This is, of course, stupidly-engineered for the purpose of the example. The next thing we have to do is to create an interface that converts calls with Objective-C conventions to calls with Python conventions:

import objc
from Foundation import NSObject
from foobar import Foobar

class PyFoobar(NSObject):
    def init(self):
        self = super(PyFoobar, self).init()
        self.py = Foobar()
        return self

    def count(self):
        return self.py.count()

    def stringAtIndex_(self, index):
        return self.py.string_at_index(index)

The signature decorators are required so that PyObjC correctly converts ints. Methods that have nothing but NSObject subclasses as arguments or return values don't need any signature.

Now that we have this, we're ready to build a plugin with py2app (the old tutorial explains how to do it). Use the interface python script as the "main plugin script" in the setup config. Once you have the plugin, you can now create your Objective-C project. The first thing you should do is to create a header file describing the interface of your Python classes:

@interface PyFoobar : NSObject {}
- (int)count;
- (NSString *)stringAtIndex:(int)aIndex;

Now comes the "magic" part where you instantiate your python classes in Objective-C. This is done through NSBundle. Let's imagine that we have a NIB-based NSWindowController with a table view that has itself as its datasource and a PyFoobar *py member. Its implementation would look like that:

- (void)awakeFromNib
    NSString *pluginPath = [[NSBundle mainBundle] pathForResource:@"your_plugin"
    NSBundle *pluginBundle = [NSBundle bundleWithPath:pluginPath];
    Class pyClass = [pluginBundle classNamed:@"PyFoobar"];
    py = [[pyClass alloc] init];

- (void)dealloc
    [py release];
    [super dealloc];

- (NSInteger)numberOfRowsInTableView:(NSTableView *)tableView
    return [py count];

- (id)tableView:(NSTableView *)tableView objectValueForTableColumn:(NSTableColumn *)column
    return [py stringAtRow:row];

You now have a working Objective-C application that embeds Python code!

I wrote earlier that I'd talk about a neat trick to reduce memory usage. In the interface code, our Foundation import will cost about 13mb (in 32-bit) of memory usage. We can consider ourselves lucky that we embed Python, or else we'd need to import NSTableView from AppKit, which would increase memory usage even further.

However, all we use is NSObject and we don't call anything on it. Couldn't we find a way to avoid that Foundation import altogether? The answer is yes. All you have to do is:

import objc
NSObject = objc.lookUpClass('NSObject')

This class, however, will have none of its methods' metadata loaded. This is not a problem for usual methods (having only NSObject arguments and return value), but for the rest of the methods, you'll need this metadata to properly call them. In this example, we don't need any metadata, but if you ever need it, know that it's possible to manually set this metadata without importing the memory-hungry Foundation and AppKit. I won't go into details here, but you can look at my own unit