edgegraph.output.nrpickler#
Non-recursive pickler / “diller” implementation to be used for serializing edgegraph objects.
This module contains a non-recursive implementation of a pickler that is
suggested for use when serializing edgegraph objects. This overcomes a
limitation of the default builtin pickler, which struggles with highly
recursive data structures – edgegraph’s internal linkage structure is indeed
very recursive, and graphs even a fraction of the recursion limit will often cause RecursionError\ s when being
pickled. This module uses dill ([dill]) to perform pickling of more
object types than the standard pickler; but even Dill uses a recursive pickler,
so needs the special handling provided here.
Danger
Like Python’s default pickler, this module is not secure. Only pickle / unpickle data you trust.
See the warning at the top of pickle documentation for more
information!
See also
The first mention of the non-recursive pickler used here was by Daniel Darabos (cyhawk) on https://bugs.python.org/issue2480 . I stole it shamelessly, only making minor changes for Python 3.x, dill compatibility, and readability / formatting. Thank you, Daniel!
As the special handling of recursive structure is only needed on the pickling
side, not the unpickling side, this module need only be used to serialize data.
Either the standard library pickle or the third-party dill
can be used to unpickle data.
Usage of this pickler should be similar to the built-in one:
>>> from edgegraph.builder import randgraph
>>> from edgegraph.output import nrpickler
>>> import pickle
>>> graph = randgraph.randgraph(count=100)
>>> serial = nrpickler.dumps(graph)
>>> serial
b'\\x80\\x04\\x8c\\x1cedgegraph.structure.universe\\x94\\x8c\\x08...'
>>> unpacked = pickle.loads(serial) # use the regular pickle module to unpack
>>> unpacked is graph
False
>>> len(unpacked.vertices) == len(graph.vertices)
True
At this point, the unpacked object is a Universe identical in every way to
graph, except it is a different instance. All of its vertices, links, and
any attributes of have all been unpacked.
Functions