Updated resuminggraph example to demonstrate useful state persistence and StateSnapshot lifecycle. Also update multiple other docs. Also added a new constructor to StateSnapshot that makes a lot of sense

Author: cpscotti

README.md

@@ -73,6 +73,9 @@ Below are a number of examples showcasing different aspects & usage patterns of
  - Dealing with Large TopicStates
     - [Portuguese Translator](@ref portuguesetranslator.cpp)
 
+ - Initial State & State Persistence
+    - [Resuming Counter](@ref resuminggraph.cpp)
+
 ## Porting
 
 The library has no dependencies beyond C++ & STL (min C++0x) and was designed to be fully portable to any environment supporting those dependencies.

doxygen/custom_examples.dox

@@ -18,6 +18,9 @@ Below are a number of examples showcasing different aspects & usage patterns of
 \section large-topicstates Dealing with Large TopicStates
     - [Portuguese Translator](@ref portuguesetranslator.cpp): @copybrief portuguesetranslator.cpp
 
+\section state-persistence Initial State & State Persistence
+    - [Resuming Counter](@ref resuminggraph.cpp): @copybrief resuminggraph.cpp
+
 */
 
 // The list of pages below is used to help the search utility finding appropriate content.

examples/resuminggraph.cpp

@@ -17,8 +17,11 @@
 #include "futurepublisher.hpp"
 #include "sharedptr.hpp"
 #include "processorcontainer.hpp"
+#include "graphstatestore.hpp"
+#include "resumefromsnapshottopicstate.hpp"
 
 #include <iostream>
+#include <fstream>
 
 using std::cout;
 using std::endl;
@@ -27,16 +30,16 @@ using std::endl;
  * @file resuminggraph.cpp
  * @brief Basic Counter Graph with persistent storage.
  *
- * @section Introduction
+ * @section ex-rg-intro Introduction
  * This examples cover the most basic way to preserve state across Graph
  * instances.
  *
  * This covers how to resume state and a neat way to define initial state.
  *
- * @section te Architecture
+ * @section ex-rg-arch Graph Architecture
  *
- * The example is composed of one Detector that counts `EventHappened`
- * and publishes a total count of those events.
+ * The example uses a minimal Graph with a single Detector and two topics.
+ * The Detector counts `EventHappened` and publishes the total in `EventCount`.
  *
  *  @dot "ResumingCounter"
 digraph GraphAnalyzer {
@@ -51,59 +54,124 @@ digraph GraphAnalyzer {
 }
  *  @enddot
  *
- * @section resume-from-snapshot ResumeFromSnapshotTopicState
- * The framework offers this convenience TopicState that easily bridges output
- * and input data without breaking the Topological sort.
- * TODO(cscotti): more.
+ * @section ex-rg-state-persistence State Persistence; Snapshots & ResumeFromSnapshotTopicState
  *
- * @section Graph
- * The graph instantiated and evaluation code is unaffected. Both
- @snippet counterwithreset.cpp CounterWithResetGraph
- * and
- @snippet counterwithreset.cpp main
- * work in the same way as with simple graphs.
+ * This example shows how [GraphStateStore](@ref DetectorGraph::GraphStateStore
+ * ) and [StateSnapshot](@ref DetectorGraph::StateSnapshot) can be used in
+ * conjunction with [ResumeFromSnapshotTopicState](@ref DetectorGraph::ResumeFromSnapshotTopicState)
+ * to provide an extensible, robust and transparent state persistence
+ * mechanism.
  *
+ * The diagram below shows the life cycle of StateSnapshots during boot.
+ *  @dot "State Lifetime FSM"
+ digraph StateLifetime {
+    node[fontname=Helvetica];
+    edge[lp="r", fontname="Times-Italic", fontcolor=blue];
+    "first_ever" [label="", shape=none];
+    "first_ever" -> "PrimeSnapshot" [label="Start"];
+    "PrimeSnapshot" [fontname=Monospace];
+    "ResumeSnapshot" [fontname=Monospace];
+    "PrimeSnapshot" -> "ResumeSnapshot" [label="Reads from Storage"];
+    "ResumeSnapshot" -> "LatestSnapshot" [label="First Graph Evaluation"];
+    subgraph clusterrunning {
+        style=filled;
+        color=lightskyblue;
+        labeljust="r";
+        label="Running & Updating GraphStateStore";
+        "LatestSnapshot" [fontname=Monospace];
+        "LatestSnapshot" -> "LatestSnapshot" [label="Subsequent Evaluations\n+\n Write to Storage"];
+    }
+ }
+ *  @enddot
+ *
+ * This life cycle can be seen in the `main` of this example:
+ @snippetlineno resuminggraph.cpp main
+ *
+ * `PrimeSnapshot` is a StateSnapshot that contains the initial state/data that
+ * should always be used - regardless of file-based state persistence.
+ * In this example we synthesize `PrimeSnapshot` this way:
+ @snippetlineno resuminggraph.cpp PrimeSnapshot
+
+ * `ResumeSnapshot` is the aggregation of TopicStates contained in
+ * `PrimeSnapshot` and what is deserialized from storage in `ReadSnapshot()`:
+ @snippetlineno resuminggraph.cpp Deserialization
+
+ * `ResumeSnapshot` is then used to construct a `ResumeFromSnapshotTopicState`
+ * which is then posted to the graph to allow Detectors to resume/initialize
+ * their state.
+ @snippetlineno resuminggraph.cpp Evaluate-ResumeFromSnapshot
+
+ * From then on `mStateStore` in `ResumingGraph` is continually updated
+ * from within `ResumingGraph::ProcessOutput()`:
+ @snippetlineno resuminggraph.cpp UpdateStateStore
+
+ * The up-to-date `latestSnapshot` can then be flushed to disk as necessary
+ * with `WriteSnapshot()`:
+ @snippetlineno resuminggraph.cpp Serialization
+
+ @note
+Different applications may chose how to do this step differently. Depending on
+your chosen Serialization technology it can be too expensive to call
+`WriteSnapshot()` for each new processed input. Some applications may choose
+to only call `WriteSnapshot()` during shutdown. A more sophisticated approach
+is to call `WriteSnapshot()` only when a `TopicState` in a set of _critical_
+ones changes. This can be done by iterating through the list in
+`mGraph.GetOutputList()` as it only contains the changed `TopicStates`.
+
+ *
+ * @section ex-rg-instance Running the Program
  * Running the program produces:
  \verbatim
+$ ./resuminggraph.out
 DetectorGraph: Graph Initialized
-EventCount.count = 1
-EventCount.count = 2
-EventCount.count = 3
-EventCount.count = 4
-EventCount.count = 5
-EventCount.count = 0
-EventCount.count = 1
-EventCount.count = 2
+EventCount = 1001
+EventCount = 1002
+EventCount = 1003
+EventCount = 1004
+EventCount = 1005
+EventCount = 1006
+EventCount = 1007
+
+$ ./resuminggraph.out
+DetectorGraph: Graph Initialized
+EventCount = 1008
+EventCount = 1009
+EventCount = 1010
+EventCount = 1011
+EventCount = 1012
+EventCount = 1013
+EventCount = 1014
+
+$ cat resumingGraphSnapshot.txt
+1014
  \endverbatim
  *
- * One important thing to note in this case is that ProcessOutput is called 8
- * times even though `main` only pushes 7 TopicStates into the graph - this is
- * by design and allows all graph outputs to continue to be inspected exactly
- * once per-evaluation.
- * This is the case for all graphs with closed loops.
+ * One important thing to note is
  *
  * @cond DO_NOT_DOCUMENT
  */
 
-//! [EventHappened]
+enum class ResumingGraphTopicStateIds{
+    kEventCount = 0,
+};
+
 struct EventHappened : public DetectorGraph::TopicState
 {
 };
-//! [EventHappened]
 
-//! [EventCount]
+//! [NamedEventCount]
 struct EventCount : public DetectorGraph::TopicState
 {
     EventCount(int aCount = 0) : count(aCount) {}
     int count;
-};
-//! [EventCount]
 
-//! [Reset]
-struct Reset : public DetectorGraph::TopicState
-{
+    DetectorGraph::TopicStateIdType GetId() const
+    {
+        return static_cast<DetectorGraph::TopicStateIdType>(
+            ResumingGraphTopicStateIds::kEventCount);
+    }
 };
-//! [Reset]
+//! [NamedEventCount]
 
 //![ResumingCountDetector]
 class ResumingCountDetector : public DetectorGraph::Detector
@@ -119,11 +187,17 @@ class ResumingCountDetector : public DetectorGraph::Detector
         SetupPublishing<EventCount>(this);
     }
 
-    void Evaluate(const DetectorGraph::ResumeFromSnapshotTopicState& aSnapshot)
+    //![Evaluate-ResumeFromSnapshot]
+    void Evaluate(const DetectorGraph::ResumeFromSnapshotTopicState& aResumeFrom)
     {
-        // TODO
-        // if (aSnapshot.ResumeFromSnapshotTopicState...)
+        const auto previousEventCount = aResumeFrom.snapshot.GetState<EventCount>();
+        if (previousEventCount)
+        {
+            mEventCount = *previousEventCount;
+        }
     }
+    //![Evaluate-ResumeFromSnapshot]
+
     void Evaluate(const EventHappened&)
     {
         mEventCount.count++;
@@ -142,29 +216,109 @@ class ResumingCountDetector : public DetectorGraph::Detector
 class ResumingGraph : public DetectorGraph::ProcessorContainer
 {
 public:
-    ResumingGraph()
+    ResumingGraph(const DetectorGraph::StateSnapshot& aInitialSnapshot)
     : mResumingCountDetector(&mGraph)
     {
-        // Post ResumeFromSnapshotTopicState
+        ProcessData(DetectorGraph::ResumeFromSnapshotTopicState(aInitialSnapshot));
     }
 
     ResumingCountDetector mResumingCountDetector;
 
     virtual void ProcessOutput()
     {
-        // update snapshot.
-        // dump to file
+        //![UpdateStateStore]
+        mStateStore.TakeNewSnapshot(mGraph.GetOutputList());
+        //![UpdateStateStore]
+    }
+
+    template<class TTopicState> ptr::shared_ptr<const TTopicState> GetLastTopicState() const
+    {
+        DG_ASSERT(DetectorGraph::TopicState::GetId<TTopicState>()
+            != DetectorGraph::TopicState::kAnonymousTopicState);
+
+        return ptr::static_pointer_cast<const TTopicState>(
+            mStateStore.GetLastState()->GetState<TTopicState>());
+    }
+    ptr::shared_ptr<const DetectorGraph::StateSnapshot> GetLastState() const
+    {
+        return mStateStore.GetLastState();
     }
+
+private:
+    DetectorGraph::GraphStateStore mStateStore;
 };
+
 //![ResumingGraph]
 
+constexpr char kSavedSnapshotFileName[] = "resumingGraphSnapshot.txt";
+
+//![PrimeSnapshot]
+DetectorGraph::StateSnapshot GetPrimeSnapshot()
+{
+    std::list< ptr::shared_ptr<const DetectorGraph::TopicState> > topicStatesList;
+    topicStatesList.push_back(ptr::shared_ptr<const DetectorGraph::TopicState>(new EventCount(1000)));
+    // ... add any other TopicStates that need Prime states.
+    return DetectorGraph::StateSnapshot(topicStatesList);
+}
+//![PrimeSnapshot]
+
+//![Serialization]
+void WriteSnapshot(const DetectorGraph::StateSnapshot& aSnapshot)
+{
+    std::ofstream snapshotOutStream(kSavedSnapshotFileName);
+
+    // Generally here you'd have a loop through a list of TopicState
+    // serializers giving each one a chance to flush that topic's contents to
+    // disk. And you'd normally use something fancier for Serialization (e.g.
+    // protobufs).
+    // For the purposes of this demo we'll mock that in a single method:
+    {
+        ptr::shared_ptr<const EventCount> countPtr = aSnapshot.GetState<EventCount>();
+        if (countPtr)
+        {
+            snapshotOutStream << countPtr->count << endl;
+        }
+        // ... add serialization of any other TopicStates of interest.
+    }
+}
+//![Serialization]
+
+//![Deserialization]
+DetectorGraph::StateSnapshot ReadSnapshot(const DetectorGraph::StateSnapshot& primeSnapshot)
+{
+    std::list< ptr::shared_ptr<const DetectorGraph::TopicState> > topicStatesList;
+
+    std::ifstream snapshotInStream(kSavedSnapshotFileName);
+    if (snapshotInStream.is_open())
+    {
+        // Generally here you'd have a loop through the TopicStates found on
+        // Storage and adding each one to `topicStatesList`
+        {
+            int count;
+            snapshotInStream >> count;
+            topicStatesList.push_back(ptr::shared_ptr<const DetectorGraph::TopicState>(new EventCount(count)));
+        }
+        // ... add deserialization of any other TopicStates of interest.
+    }
+
+    return DetectorGraph::StateSnapshot(primeSnapshot, topicStatesList);
+}
+//![Deserialization]
+
 //![main]
 int main()
 {
-    ResumingGraph resumingGraph;
+    DetectorGraph::StateSnapshot primeSnapshot = GetPrimeSnapshot()
+    DetectorGraph::StateSnapshot resumeSnapshot = ReadSnapshot(primeSnapshot);
+
+    ResumingGraph resumingGraph = ResumingGraph(resumeSnapshot);
     for (int i = 0; i < 7; ++i)
     {
         resumingGraph.ProcessData(EventHappened());
+        cout << "EventCount = " << resumingGraph.GetLastTopicState<EventCount>()->count << endl;
+
+        const auto latestSnapshot = resumingGraph.GetLastState();
+        WriteSnapshot(*latestSnapshot);
     }
 }
 //![main]

include/statesnapshot.hpp

@@ -41,6 +41,13 @@ class StateSnapshot
      */
     StateSnapshot();
 
+    /**
+     * @brief T=0 Priming Constructor
+     *
+     * Builds a prime StateSnapshot from a TopicState list.
+     */
+    StateSnapshot(const std::list< ptr::shared_ptr<const TopicState> >& arTopicStates);
+
     /**
      * @brief T>0 Constructor
      *

src/statesnapshot.cpp

@@ -21,8 +21,16 @@ namespace DetectorGraph
 {
 
 StateSnapshot::StateSnapshot()
+: mStateStore()
+, mStateVersion(0)
 {
-    mStateVersion = 0;
+}
+
+StateSnapshot::StateSnapshot(const std::list< ptr::shared_ptr<const TopicState> >& arTopicStates)
+: mStateStore()
+, mStateVersion(0)
+{
+    UpdateValues(arTopicStates);
 }
 
 StateSnapshot::StateSnapshot(const StateSnapshot& arPreviousState, const std::list< ptr::shared_ptr<const TopicState> >& arTopicStates)