◐ Shell
clean mode source ↗

Message 186705 - Python tracker

Python 2 allows pickling and unpickling non-ascii persistent ids. In Python 3 C implementation of pickle saves persistent ids with protocol version 0 as utf8-encoded strings and loads as bytes.

>>> import pickle, io
>>> class MyPickler(pickle.Pickler):
...     def persistent_id(self, obj):
...         if isinstance(obj, str):
...             return obj
...         return None
... 
>>> class MyUnpickler(pickle.Unpickler):
...     def persistent_load(self, pid):
...         return pid
... 
>>> f = io.BytesIO(); MyPickler(f).dump('\u20ac'); data = f.getvalue()
>>> MyUnpickler(io.BytesIO(data)).load()
'€'
>>> f = io.BytesIO(); MyPickler(f, 0).dump('\u20ac'); data = f.getvalue()
>>> MyUnpickler(io.BytesIO(data)).load()
b'\xe2\x82\xac'
>>> f = io.BytesIO(); MyPickler(f, 0).dump('a'); data = f.getvalue()
>>> MyUnpickler(io.BytesIO(data)).load()
b'a'

Python implementation in Python 3 doesn't works with non-ascii persistant ids at all.