(Un)Pickle Class having Instancemethod Objects

Question

I have a class (Bar) which effectively has its own state and callback(s) and is used by another class (Foo):

 class Foo(object):

     def __init__(self):
         self._bar = Bar(self.say, 10)
         self._bar.work()

     def say(self, msg):
         print msg

 class Bar(object):
     def __init__(self, callback, value):
         self._callback = callback
         self._value = value
         self._more = { 'foo' : 1, 'bar': 3, 'baz': 'fubar'}

     def work(self):
         # Do some work
         self._more['foo'] = 5
         self._value = 10
         self._callback('FooBarBaz')

 Foo()

Obviously I can't pickle the class Foo since Bar has an instancemethod, so I'm left with the following solution of implementing __getstate__ & __setstate__ in Bar to save self._value & self._more, but I have to instantiate the self._callback method as well (i.e. call __init__() from the outer class Foo passing the callback function.

But I cannot figure out how to achieve this.

Any help is much appreciated.
Thanks.

Answer 1

I think if you need to serialize something like this you need to be able to define your callback as a string. For example, you might say that callback = 'myproject.callbacks.foo_callback'.

Basically in __getstate__ you'd replace the _callback function with something you could use to look up the function later like self._callback.__name__.

In __setstate__ you'd replace _callback with a function.

This depends on your functions all having real names so you couldn't use a lambda as a callback and expect it to be serialized. You'd also need a reasonable mechanism for looking up your functions by name.

You could potentially use __import__ (something like: 'myproject.somemodule.somefunc' dotted name syntax could be supported that way, see http://code.google.com/p/mock/source/browse/mock.py#1076) or just define a lookup table in your code.

Just a quick (untested, sorry!) example assuming you have a small set of possible callbacks defined in a lookup table:

def a():
    pass

callbacks_to_name = {a: 'a'
                     # ...
                     }

callbacks_by_name = {'a': a,
             # ...
             }

class C:
    def __init__(self, cb):
        self._callback = cb

    def __getstate__(self):
        self._callback = callbacks_to_name[self._callback]
        return self.__dict__

    def __setstate__(self, state):
        state[_callback] = callbacks_by_name[self._callback]

I'm not sure what your use case is but I'd recommend doing this by serializing your work items to JSON or XML and writing a simple set of functions to serialize and deserialize them yourself.

The benefit is that the serialized format can be read and understood by humans and modified when you upgrade your software. Pickle is tempting because it seems close enough, but by the time you have a serious pile of __getstate__ and __setstate__ you haven't really saved yourself much effort or headache over building your own scheme specifically for your application.