Issue 32696: Fix pickling exceptions with multiple arguments

Pickling exceptions fails when a custom exception class requires multiple arguments.

import pickle
class MultipleArgumentsError(Exception):
    def __init__(self, a, b):
        self.a = a
        self.b = b

pickle.loads(pickle.dumps(MultipleArgumentsError('a', 'b')))

this code produces the following error:

Traceback (most recent call last):
  File "/tmp/multiple_arguments_exception.py", line 8, in <module>
    pickle.loads(pickle.dumps(MultipleArgumentsError('a', 'b')))
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 1388, in loads
    return Unpickler(file).load()
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 864, in load
    dispatch[key](self)
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 1139, in load_reduce
    value = func(*args)
TypeError: __init__() takes exactly 3 arguments (2 given)


The same error occurs when using a built-in module like subprocess:
>>> import subprocess, pickle
>>> try:
...   subprocess.check_call(['python', '-c', 'raise SystemExit(1)'])
... except Exception as e:
...   pickle.loads(pickle.dumps(e))
... 


Related issue: https://bugs.python.org/issue1692335

Hi, I encounter similar behavior in python 3.6.5 with following code:

import pickle
class CustomException(Exception):
    def __init__(self, arg1, arg2):
        msg = "Custom message {} {}".format(arg1, arg2)
        super().__init__(msg)


obj_dump = pickle.dumps(CustomException("arg1", "arg2"))
obj = pickle.loads(obj_dump)

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: __init__() missing 1 required positional argument: 'arg2'

So it looks like it not only python 2.7 problem

Kirill, see https://bugs.python.org/issue1692335#msg310951 in the related issue for one possible way to work around the issue on Python 3.

This is still the same in 3.11:

>>> import pickle
>>> class CustomException(Exception):
...     def __init__(self, arg1, arg2):
...         msg = "Custom message {} {}".format(arg1, arg2)
...         super().__init__(msg)
...
>>> obj_dump = pickle.dumps(CustomException("arg1", "arg2"))
>>> obj = pickle.loads(obj_dump)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: CustomException.__init__() missing 1 required positional argument: 'arg2'
>>>

It seems like the example in the OP now works (persist both arguments), but Irit's/Kirill's (create a msg) fails (I am running 3.9).

See also issue43460, issue30005, issue29466

This isn't really an issue with exceptions, it's just that __reduce__ goes out of sync with the object's constructor signature. Here's a simple example of the same:

class Base:
   def __init__(self, msg):
       self.msg = msg

   def __reduce__(self):
       return type(self), (self.msg,)

class Derived(Base):
   def __init__(self, a, b):
       super().__init__(f'{a}|{b}')

x = Derived('a', 'b')
assert x.msg == 'a|b'
y = pickle.dumps(x, -1)
z = pickle.loads(y)

-------------------------------------------
Output:

Traceback (most recent call last):
  File "/Users/iritkatriel/src/cpython/zz.py", line 22, in <module>
    z = pickle.loads(y)
        ^^^^^^^^^^^^^^^
TypeError: Derived.__init__() missing 1 required positional argument: 'b'

If I define __reduce__ on Derived to return an arg tuple of the right length for its __init__, it works:

class Derived(Base):
   def __init__(self, a, b):
       super().__init__(f'{a}|{b}')

   def __reduce__(self):
       return type(self), tuple(self.msg.split('|'))

But note that this is not something we could make the base class do, it has no way of finding out what the semantics of the derived class constructor args are.

pull_requests: + pull_request28801

messages: + msg396035
versions: + Python 3.11, - Python 2.7