edgegraph.output.nrpickler

Non-recursive pickler / “diller” implementation to be used for serializing edgegraph objects.

This module contains a non-recursive implementation of a pickler that is suggested for use when serializing edgegraph objects. This overcomes a limitation of the default builtin pickler, which struggles with highly recursive data structures – edgegraph’s internal linkage structure is indeed very recursive, and graphs even a fraction of the recursion limit will often cause RecursionErrors when being pickled. Even dill uses a recursive implementation, so this is necessary for use with dill as well.

Danger

Like Python’s default pickler, this module is not secure. Only pickle / unpickle data you trust.

See the warning at the top of pickle documentation for more information!

See also

The first mention of the non-recursive pickler used here was by Daniel Darabos (cyhawk) on https://bugs.python.org/issue2480 . I stole it shamelessly, only making minor changes for Python 3.x, dill compatibility, and readability / formatting. Thank you, Daniel!

Usage of this pickler should be similar to the built-in one:

>>> from edgegraph.builder import randgraph
>>> from edgegraph.output import nrpickler
>>> import pickle
>>> graph = randgraph.randgraph(count=100)
>>> serial = nrpickler.dumps(graph)
>>> serial
b'\x80\x04\x8c\x1cedgegraph.structure.universe\x94\x8c\x08...'
>>> unpacked = pickle.loads(serial)  # use the regular pickle module to unpack
>>> unpacked is graph
False
>>> len(unpacked.vertices) == len(graph.vertices)
True

At this point, the unpacked object is a Universe identical in every way to graph, except it is a different instance. All of its vertices, links, and any attributes of have all been unpacked.

Functions

dump(obj, file[, protocol, byref, fmode, ...])	Module-level interface to the non-recursive pickler dump.
dumps(obj[, protocol, byref, fmode, recurse])	Module-level interface to the non-recursive pickler dumps.

Classes

NonrecursivePickler(file, **kwargs)

Non-recursive pickler class.