Added Testing Style and Tips document. (#9)

Author: cpscotti

doxygen/style_suggestions.dox

@@ -36,13 +36,13 @@ There are two ways of accomplishing this; composition or inheritance.
 Composition is often preferable over inheritance as it gives more flexibility but both should be considered and are possible.
 
 Composition example (from [Robot Localization](@ref robotlocalization.cpp)):
-@snippet robotlocalization.cpp TopicStates Composition Example
+@snippetlineno robotlocalization.cpp TopicStates Composition Example
 
 Inheritance examples
 From [Fancy Vending Machine](@ref fancyvendingmachine.cpp):
-@snippet fancyvendingmachine.cpp TopicStates Inheritance Example
+@snippetlineno fancyvendingmachine.cpp TopicStates Inheritance Example
 From [Beat Machine](@ref beatmachine.cpp):
-@snippet beatmachine.cpp TopicStates Inheritance Example
+@snippetlineno beatmachine.cpp TopicStates Inheritance Example
 
 Inheritance can save you some typing and be OK when literally all you're changing is the type 'name' and when you know the data structure will never change and when the 'is a' relationship holds strongly. Inheritance is the lazy approach but that you'll likely have to ditch (at great expense possibly) at some point in the future.
 
@@ -51,19 +51,19 @@ Inheritance can save you some typing and be OK when literally all you're changin
 Sometimes all a topic needs to express is the fact that _it updated_, that it _happened_ or any other purely unary signals:
 
 From [Trivial Vending Machine](@ref trivialvendingmachine.cpp):
-@snippet trivialvendingmachine.cpp Trivial TopicState
+@snippetlineno trivialvendingmachine.cpp Trivial TopicState
 From [Fancy Vending Machine](@ref fancyvendingmachine.cpp):
-@snippet fancyvendingmachine.cpp Trivial TopicState
+@snippetlineno fancyvendingmachine.cpp Trivial TopicState
 
 \subsection ssg-atomic-change Variables/bits/data that change together atomically belong together.
 
 Examples:
 
 A product and its new price:
-@snippet fancyvendingmachine.cpp Mutually Atomic Variables
+@snippetlineno fancyvendingmachine.cpp Mutually Atomic Variables
 
 A robot's pose and a timestamp:
-@snippet robotlocalization.cpp Mutually Atomic Variables
+@snippetlineno robotlocalization.cpp Mutually Atomic Variables
 
 \subsection ssg-mutually-exclusive Mutually exclusive states also belong together.
 

doxygen/testing_style.dox

@@ -0,0 +1,101 @@
+/**
+@page testing-style Testing - Unit & Integration Tests
+
+\tableofcontents
+
+\section ts-unit Unit Tests
+
+One of the design goals of the Framework was to facilitate full testing coverage of any detectors. DetectorGraph is designed so that each Detector is tested as a Unit - by passing specific input `TopicStates` and comparing output `TopicStates` with expected values.
+@warning Before continuing it may be good to familiarize yourself with the first few [examples](@ref custom_examples) as this document builds on top of those examples.
+
+\subsection ts-unit-siso Basics
+
+The most basic case is for a SISO Detector like the one in the [HelloWorld Example](@ref helloworld.cpp). Detector unit tests should be written following the Arrange-Act-Assert (AAA) unit testing pattern.
+
+_Arrange_ for the test consists of a graph instance with only the detector under test in it. In addition to that it's also cursory to grab a pointer `outTopic` to the output Topics we will check in the Assert section of the test. This help setting expectations for the reader and cleans up the later sections of the test.
+@snippetlineno helloworld.cpp UnitTest-Above-1
+Note that different Unit Testing frameworks will use different function signatures for tests so adapt it as necessary.
+
+_Act_ consists in _Pushing_ the input data topic into the input queue and _Evaluating_ the graph.
+@snippetlineno helloworld.cpp UnitTest-Above-2
+
+_Assert_ consists in [inspecting](@ref topic_inspection_apis) the output topic(s):
+@snippetlineno helloworld.cpp UnitTest-Above-3
+Different Unit Testing frameworks will use different & more expressive assert methods but the idea is the same.
+
+\subsection ts-unit-siso-aaa-aa Act-Assert sequences
+
+In some cases it's nice to submit a detector to a small sequence of Act/Asserts. Care should be taken to maintain the test focused on a single behavior and not let it balloon out of proportion (testing more than one single behavior) - so this only applies when the behavior being tested is sequential in nature.
+
+One such case is to test the counting behavior of the [CounterWithReset Example](@ref counterwithreset.cpp).
+The test starts as before:
+@snippetlineno counterwithreset.cpp UnitTest-Count-1
+
+And follows with an extra _Act_ and _Assert_ pair:
+@snippetlineno counterwithreset.cpp UnitTest-Count-2
+Note that the middle _Assert_ is not necessary per se - it is helpful when it gives the reader confidence that _things are going as expected_ but they will be *very* unhelpful if they distract the reader from what's coming next; one or a couple of assert lines are fine, 10 are not. Use your best judgment here.
+
+\subsection ts-unit-miso Multiple Input Single Output
+
+Most times when testing a detector with multiple inputs and multiple outputs each input has to be _Pushed_ and _Evaluated_ on its own.
+Again, for the [CounterWithReset Example](@ref counterwithreset.cpp):
+@snippetlineno counterwithreset.cpp UnitTest-ResetCount-1
+Here the _Arrange_ step includes pushing some data and evaluating the graph.
+
+_Act_ and _Assert_ are then done just as before:
+@snippetlineno counterwithreset.cpp UnitTest-ResetCount-2
+
+\subsection ts-unit-futpub Future Publishes
+
+In order for detectors to close feedback loops/cycles in the graph they must be [FuturePublisher](@ref DetectorGraph::FuturePublisher)s.
+The simplest example of this is shown in the `ResetDetector` within the [CounterWithReset Example](@ref counterwithreset.cpp).
+The interesting detail here is that Topics published with [PublishOnFutureEvaluation](@ref DetectorGraph::FuturePublisher::PublishOnFutureEvaluation) are, as the name suggests, published only in the next evaluation and that needs to be taken into account in the unit test.
+
+@warning See the section below for a note on [Lag](@ref DetectorGraph::Lag) for reasons why that's preferred than what's shown here.
+
+Initial _Arrange_ and _Act_ are as normal:
+@snippetlineno counterwithreset.cpp UnitTest-ResetDetected-1
+
+But the test must check:
+ - that the output is not immediately published
+ - that the intended `TopicState` is published on the next evaluation.
+
+And that can be done with:
+@snippetlineno counterwithreset.cpp UnitTest-ResetDetected-2
+
+The same idea applies when testing the negative complementary of such behaviors:
+@snippetlineno counterwithreset.cpp UnitTest-NoResetDetected-1
+
+\subsection ts-unit-lag Lag
+
+[Lags](@ref DetectorGraph::Lag) are the more general, preferred & readable way to close loops in a graph - and one place they shine the shiniest is in unit tests; since Lags are Detectors themselves and the lagged Topic is just another Topic. That means unit tests do not need to keep track of future publishes as that's all handled outside that one detector. Unit tests then are trivial with respect to the loop:
+
+@snippetlineno fancyvendingmachine.cpp UnitTest-LowerUserBalanceOnSale
+
+@note There are two reasons why Lags didn't simply replace `FuturePublish`; legacy code uses FuturePublish extensively and Lags may a bit wasteful in deeply embedded (limited code space) targets.
+
+
+\section ts-system Integration Tests
+
+In order to test the integration between detectors one can instantiate a subset of detectors (up to the entire set) of an application and test it in the same way as unit tests - all you have to do is think of the whole thing as one single big detector. This test should then be stable regardless of how many detectors compose the solution in between the inputs and outputs.
+
+In integration tests it's important to _flush_ the graph input queue continuously when appropriate for the reasons described in @ref ts-unit-futpub . This can be seen in the example for [CounterWithReset](@ref counterwithreset.cpp) below on line 311:
+
+@snippetlineno counterwithreset.cpp UnitTest-CounterResetIntegration
+
+*/
+
+/**
+@page seo-testing Testing
+See [Testing Style](@ref testing-style).
+*/
+
+/**
+@page seo-unit-testing Unit Testing
+@copydoc seo-testing
+*/
+
+/**
+@page seo-integ-testing Integration Testing
+@copydoc seo-testing
+*/

examples/counterwithreset.cpp

@@ -163,6 +163,49 @@ class EventCountDetector : public DetectorGraph::Detector
 };
 //![EventCountDetector]
 
+//![UnitTest-Count-1]
+void Test_Count()
+{
+    DetectorGraph::Graph graph;
+    EventCountDetector detector(&graph);
+    auto outTopic = graph.ResolveTopic<EventCount>();
+
+    graph.PushData(EventHappened());
+    graph.EvaluateGraph();
+
+    DG_ASSERT(outTopic->HasNewValue());
+    DG_ASSERT(outTopic->GetNewValue().count == 1);
+
+    //![UnitTest-Count-1]
+    //![UnitTest-Count-2]
+    graph.PushData(EventHappened());
+    graph.EvaluateGraph();
+
+    DG_ASSERT(outTopic->HasNewValue());
+    DG_ASSERT(outTopic->GetNewValue().count == 2);
+}
+//![UnitTest-Count-2]
+
+//![UnitTest-ResetCount-1]
+void Test_ResetCount()
+{
+    // Arrange
+    DetectorGraph::Graph graph;
+    EventCountDetector detector(&graph);
+    auto outTopic = graph.ResolveTopic<EventCount>();
+    graph.PushData(EventHappened());
+    graph.EvaluateGraph();
+
+    //![UnitTest-ResetCount-1]
+    //![UnitTest-ResetCount-2]
+    graph.PushData(Reset());
+    graph.EvaluateGraph();
+
+    DG_ASSERT(outTopic->HasNewValue());
+    DG_ASSERT(outTopic->GetNewValue().count == 0);
+}
+//![UnitTest-ResetCount-2]
+
 //![Reset Detector]
 class ResetDetector : public DetectorGraph::Detector
 , public DetectorGraph::SubscriberInterface<EventCount>
@@ -187,6 +230,43 @@ class ResetDetector : public DetectorGraph::Detector
 };
 //![Reset Detector]
 
+//![UnitTest-ResetDetected-1]
+void Test_ResetDetected()
+{
+    // Arrange
+    DetectorGraph::Graph graph;
+    ResetDetector detector(&graph);
+    auto outTopic = graph.ResolveTopic<Reset>();
+    graph.PushData(EventCount(ResetDetector::kMaxCount));
+    graph.EvaluateGraph();
+
+    //![UnitTest-ResetDetected-1]
+    //![UnitTest-ResetDetected-2]
+    DG_ASSERT(!outTopic->HasNewValue());
+
+    graph.EvaluateGraph();
+
+    DG_ASSERT(outTopic->HasNewValue());
+}
+//![UnitTest-ResetDetected-2]
+
+//![UnitTest-NoResetDetected-1]
+void Test_NoResetDetected()
+{
+    // Arrange
+    DetectorGraph::Graph graph;
+    ResetDetector detector(&graph);
+    auto outTopic = graph.ResolveTopic<Reset>();
+    graph.PushData(EventCount(ResetDetector::kMaxCount-1));
+    graph.EvaluateGraph();
+
+    DG_ASSERT(!outTopic->HasNewValue());
+
+    graph.EvaluateGraph();
+    DG_ASSERT(!outTopic->HasNewValue());
+}
+//![UnitTest-NoResetDetected-1]
+
 //![CounterWithResetGraph]
 class CounterWithResetGraph : public DetectorGraph::ProcessorContainer
 {
@@ -213,6 +293,31 @@ class CounterWithResetGraph : public DetectorGraph::ProcessorContainer
 };
 //![CounterWithResetGraph]
 
+//![UnitTest-CounterResetIntegration]
+void Test_CounterResetIntegration()
+{
+    DetectorGraph::Graph graph;
+    EventCountDetector counterDetector(&graph);
+    ResetDetector resetDetector(&graph);
+    auto outTopic = graph.ResolveTopic<EventCount>();
+
+    for (int i = 1; i <= ResetDetector::kMaxCount; i++)
+    {
+        graph.PushData(EventHappened());
+        graph.EvaluateGraph();
+        DG_ASSERT(outTopic->HasNewValue());
+        DG_ASSERT(outTopic->GetNewValue().count == i);
+
+        while(graph.EvaluateIfHasDataPending()) {} // See also GraphTestUtils::Flush()
+    }
+
+    graph.PushData(EventHappened());
+    graph.EvaluateGraph();
+    DG_ASSERT(outTopic->HasNewValue());
+    DG_ASSERT(outTopic->GetNewValue().count == 1);
+}
+//![UnitTest-CounterResetIntegration]
+
 //![main]
 int main()
 {
@@ -221,6 +326,12 @@ int main()
     {
         counterGraph.ProcessData(EventHappened());
     }
+
+    Test_Count();
+    Test_ResetCount();
+    Test_ResetDetected();
+    Test_NoResetDetected();
+    Test_CounterResetIntegration();
 }
 //![main]
 

examples/fancyvendingmachine.cpp

@@ -540,6 +540,25 @@ class UserBalanceDetector : public DetectorGraph::Detector
 
 };
 
+//![UnitTest-LowerUserBalanceOnSale]
+void Test_LowerUserBalanceOnSale()
+{
+    DetectorGraph::Graph graph;
+    UserBalanceDetector detector(&graph);
+    auto outTopic = graph.ResolveTopic<UserBalance>();
+    graph.PushData(CoinInserted(kCoinType1d));
+    graph.EvaluateGraph();
+    DG_ASSERT(outTopic->HasNewValue());
+    DG_ASSERT(outTopic->GetNewValue().totalCents == 100);
+
+    graph.PushData(Lagged<SaleProcessed>(SaleProcessed(kProductIdTypeNone, 75)));
+    graph.EvaluateGraph();
+
+    DG_ASSERT(outTopic->HasNewValue());
+    DG_ASSERT(outTopic->GetNewValue().totalCents == 25);
+}
+//![UnitTest-LowerUserBalanceOnSale]
+
 class SaleProcessor : public DetectorGraph::Detector
 , public DetectorGraph::SubscriberInterface<UserBalance>
 , public DetectorGraph::SubscriberInterface<SelectedProduct>
@@ -854,6 +873,8 @@ int main()
 
     GraphAnalyzer analyzer(fancyVendingMachine.mGraph);
     analyzer.GenerateDotFile("fancy_vending_machine.dot");
+
+    Test_LowerUserBalanceOnSale();
 }
 
 /// @endcond DO_NOT_DOCUMENT

examples/helloworld.cpp

@@ -194,6 +194,44 @@ class HelloWorldGraph : public DetectorGraph::ProcessorContainer
 };
 //![ProcessorContainer]
 
+//![UnitTest-Above-1]
+void Test_AboveThreshold()  // Adapt to your Unit Test Framework
+{
+    // Arrange
+    DetectorGraph::Graph graph;
+    OverheatingDetector detector(&graph);
+    auto outTopic = graph.ResolveTopic<OverheatingState>();
+    //![UnitTest-Above-1]
+    //![UnitTest-Above-2]
+
+    // Act
+    graph.PushData(TemperatureSample(OverheatingDetector::kThreshold+1));
+    graph.EvaluateGraph();
+
+    //![UnitTest-Above-2]
+    //![UnitTest-Above-3]
+    // Assert
+    DG_ASSERT(outTopic->HasNewValue()); // Adapt to your Unit Test Framework
+    DG_ASSERT(outTopic->GetNewValue().isOverheating == true); // Adapt to your Unit Test Framework
+}
+//![UnitTest-Above-3]
+
+void Test_BelowThreshold()
+{
+    // Arrange
+    DetectorGraph::Graph graph;
+    OverheatingDetector detector(&graph);
+    auto outTopic = graph.ResolveTopic<OverheatingState>();
+
+    // Act
+    graph.PushData(TemperatureSample(OverheatingDetector::kThreshold-1));
+    graph.EvaluateGraph();
+
+    // Assert
+    DG_ASSERT(outTopic->HasNewValue());
+    DG_ASSERT(outTopic->GetNewValue().isOverheating == false);
+}
+
 //![main]
 int main()
 {
@@ -222,6 +260,11 @@ int main()
     thermostat.ProcessData(TemperatureSample(110));
     thermostat.ProcessData(TemperatureSample(120));
     //![Using ProcessorContainer]
+
+    // Normally your Unit test framework of choice would call this
+    // automatically. We do explicitly for demonstration purposes.
+    Test_AboveThreshold();
+    Test_BelowThreshold();
 }
 //![main]