Picklable exceptions in Python

Let’s say, we have simple exception like this:

>>> class MyException(Exception):
...
...     def __init__(self, arg):
...         pass
...

Let’s take a look at output of pickle.dumps:

>>> from pickle import dumps, loads
>>> print dumps(MyException(54))
c__main__
MyException
p0
(tRp1
.

Seems good? What if we’ll define another exception:

>>> class MyException2(Exception):
...
...     def __init__(self, arg, arg2):
...         pass
...

And look at output of dumps:

>>> print dumps(MyException2(9, 8))
c__main__
MyException2
p0
(tRp1
.

Nothing changed except name of exception class. And both of these strings will cause exception like this:

Traceback (most recent call last):
  File "...", line 1, in <module>
    loads(dumps(MyException(54)))
  File "/usr/lib/python2.7/pickle.py", line 1382, in loads
    return Unpickler(file).load()
  File "/usr/lib/python2.7/pickle.py", line 858, in load
    dispatch[key](self)
  File "/usr/lib/python2.7/pickle.py", line 1133, in load_reduce
    value = func(*args)
TypeError: __init__() takes exactly 2 arguments (1 given)

Actually, for MyException2 it will say __init__() takes exactly 3 arguments, as it takes two arguments we defined in addition to self.

This is definitely a problem. But this can be easily fixed: we need to add Exception.__init__ call into our __init__ and look at pickle.dumps output again:

>>> print dumps(MyException(54))
c__main__
MyException
p0
(I54
tp1
Rp2
.

As we can see, this time there is actually 54 in output string. And pickle.loads works too:

>>> loads(dumps(MyException(54)))
MyException(54,)

To make exception (MyException), that accepts *args, work with pickle, we added Exception.__init__(self, *args) into MyException.__init__. By the way, there is no support for keyword arguments in exceptions.

Another way to make exceptions picklable is to set self.args to tuple of arguments manually:

>>> class E(Exception):
...
...     def __init__(self, a):
...         self.args = (a, )
...

Let’s check if it works:

>>> print dumps(E(31337))
c__main__
E
p0
(I31337
tp1
Rp2
.

>>> loads(dumps(E(31337)))
E(31337,)