Boost Serialization support for Variable Length, Dynamically Allocated data.

[Joss Whittle]

Hello, we are currently working on using Boost Serialization to read and write generic graph structures to/from file.

Below we have a simple & generic directed graph container. Our goal is to serialize this structure to file. We do not know ahead of serialization / de-serialization what the full shape of the graph is. In both directions we only have the graph container.

template<typename T>
struct DiGraphNode {
    /// Members
    T value;
    std::vector<DiGraphNode<T>*> edges;

    DiGraphNode() {};
    DiGraphNode(const T &_v) : value(_v) {};

    inline int getOutDegree() const {
        return (int) edges.size();
    }

    inline void addEdge(DiGraphNode<T> *node) {
        edges.push_back(node);
    }

    template<class Archive>
    void serialize(Archive& ar, const unsigned version) {

        /// Help

    }
};

template<typename T>
struct DiGraph {
    /// Members
    std::vector<DiGraphNode<T>*> nodes;

    DiGraph() {};
    ~DiGraph() {
        for (auto it = nodes.begin(); it != nodes.end(); ++it) {
            delete *it;
        }
    }

    inline void addNode(const T &value) {
        nodes.push_back(new DiGraphNode<T>(value));
    }

    inline DiGraphNode<T>* getNode(const int nodeId) const {
        return nodes[nodeId];
    }

    template<class Archive>
    void serialize(Archive& ar, const unsigned version) {

        // Help

    }
};

We create an empty graph and add nodes / edges randomly to create our test structure. This is a graph where multiple nodes can point to the same node, including themselves.

DiGraph<int>* MakeRandomGraph(const int numNodes) {
    srand(1234567);

    /// Random Graph
    DiGraph<int> *graph = new DiGraph<int>();

    /// Create graph nodes
    for (int i = 0; i < numNodes; ++i) {
        graph->addNode(i);
    }

    /// Connect each node to a random set of neighbours
    for (int i = 0; i < numNodes; ++i) {
        const int numEdges = rand() % numNodes;

        DiGraphNode<int> *node = graph->getNode(i);
        for (int j = 0; j < numEdges; ++j) {
            node->addEdge(graph->getNode(rand() % numNodes));
        }
    }
    return graph;
}

Here is an example use case. We create a random graph and serialize it to file. Then, somewhere else, we open the file and read the graph back in. How do we tell the serialization function how to allocate nodes that have not been visited yet, and to reuse nodes if they have already been visited.

/// Create a graph for testing
DiGraph<int> *graph = MakeRandomGraph(32);

/// Write the graph to file
std::ofstream graphFile("random.graph", std::ios::out | std::ios::binary);
if (graphFile.is_open()) {
    boost::archive::binary_oarchive oa(graphFile);
    oa << *graph;
    graphFile.close();
}
delete graph;

/// Later - Load the graph back in from file
DiGraph<int> *graph = nullptr;

/// Reconstruct the graph from file
std::ifstream graphFile("random.graph", std::ios::in | std::ios::binary);
if (graphFile.is_open()) {
    boost::archive::binary_iarchive ia(graphFile);
    graph = new DiGraph<int>();
    ia >> *graph;
    graphFile.close();
}

Any support would be great. We have poured over the documentation and communities such as stack overflow but have not been able to find advice on such variable and shared allocations.

Thanks,
Joss Whittle