This is the 26th day of my participation in the August Text Challenge.More challenges in August

Hardware and software Environment

  • windows 10 64bits
  • Anaconda with python 3.7
  • pickle

Introduction to the

The pickle module implements binary serialization and deserialization of a Python object structure. Pickling is the process of converting a Python object and its own hierarchy into a byte stream, while unpickling is the opposite operation, converting the byte stream back into an object hierarchy.

Almost every data type in Python (lists, dictionaries, collections, classes, etc.) can be serialized with pickle.

Common methods for pickling

The pickle module provides the following methods to make the serialization and deserialization process easier

  • Dump method

    pickle.dump(obj, file, protocol=None, *, fix_imports=True, buffer_callback=None)
Copy the code

    Writes the serialized object obj to an open file object. The protocol parameter is serialization mode, with a default value of 0 indicating serialization as text. Protocol can also have a value of 1 or 2, indicating serialization in binary form.

  • Dumps method

    pickle.dumps(obj, protocol=None, *, fix_imports=True, buffer_callback=None)
Copy the code

    The obj sealed object is returned directly as a bytes type, rather than being written to a file object. Parameters are the same as those in dump.

  • The load method

    pickle.load(file, *, fix_imports=True, encoding="ASCII", errors="strict", buffers=None)
Copy the code

    Reads the serialized object from an open file object, reconstructs the hierarchy of the particular object in it, and returns.

    The pickle protocol version is automatically detected, so no parameters are required to specify the protocol. Bytes other than the sealed object are ignored.

  • Loads method

    pickle.loads(data, /, *, fix_imports=True, encoding="ASCII", errors="strict", buffers=None)
Copy the code

    Rebuild and return the object hierarchy of data. Data is a serialized Bytes object.

The sample code

First, take a look at the serialization process, serializing strings, dictionaries, and lists, respectively, using the dump method

PS C:\Users\Administrator> ipython Python 3.7.6 (default, Jan 8 2020, 20:23:39) [MSC v.1916 64 bit (AMD64)] Type 'copyright', 'Credits' or 'license' for more information IPython 7.12.0 -- An enhanced Interactive python. Type '? ' for help. In [1]: aString = 'Hello pickle' In [2]: aDict = {'p': 'python', 'r': 'rust', 's': 'swift'} In [3]: aList = ['one', 'two', 'three'] In [5]: f = open('test.pkl', 'wb') In [6]: pickle.dump(aString, f, True) <IPython.core.display.Javascript object> In [7]: pickle.dump(aDict, f, True) <IPython.core.display.Javascript object> In [8]: pickle.dump(aList, f, True) <IPython.core.display.Javascript object> In [9]: f.close() In [10]:Copy the code

At this point, the binary test.pkl is generated under the directory C: Users\Administrator

Next, deserialize

In [11]: f = open('test.pkl', 'wb')

In [12]: pickle.dump(aString, f, True)
<IPython.core.display.Javascript object>

In [13]: pickle.dump(aDict, f, True)
<IPython.core.display.Javascript object>

In [14]: pickle.dump(aList, f, True)
<IPython.core.display.Javascript object>

In [15]: f.close()

In [16]: f1 = open('test.pkl', 'rb')

In [17]: lString = pickle.load(f1)
<IPython.core.display.Javascript object>

In [18]: lString
Out[18]: 'Hello pickle'

In [19]: lDict = pickle.load(f1)
<IPython.core.display.Javascript object>

In [20]: lDict
Out[20]: {'p': 'python', 'r': 'rust', 's': 'swift'}

In [21]: lList = pickle.load(f1)
<IPython.core.display.Javascript object>

In [22]: lList
Out[22]: ['one', 'two', 'three']

In [23]: f1.close()
Copy the code

From the above code, you can see that deserialization takes place in the same order as serialization, much like queues in data structures.

File objects cannot be serialized

Python cannot serialize a file object, or any object with a reference to a file object, because there is no guarantee that it can recreate the state of the file when deserializing it. Look at the following example

In [32]: f = open('test.pkl', 'wb')

<IPython.core.display.Javascript object>
In [33]: p = pickle.dumps(f)
<IPython.core.display.Javascript object>
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-33-671076828a70> in <module>
----> 1 p = pickle.dumps(f)

TypeError: cannot serialize '_io.BufferedWriter' object

In [34]:
Copy the code

portability

The pickle file format is machine architecture-independent, which means that you can create a pickle on Linux and send it to a Python program running on Windows or MacOS. Also, when the Python version is upgraded, there is no need to worry about existing pickle operations, which can be backward compatible.