#2 Regression test suite needs to be updated for all implemented algorithms

[Bryan Thompson]

The performance and correctness regressions/ directory needs to be updated to capture (a) all algorithms and (b) all implementations (CUDA, OpenMP, TBB) [1].

• Powergraph is difficult to install. We need to setup a machine on which powergraph is correctly installed for regression testing in CI.

• We need to verify that the regression tests run correctly for the different mpgraph algorithms when compared to powergraph. (As part of this, verify that the same starting vertex is used for the powergraph and mpgraph implementations. The original code used the vertex with the highest degree, but the current code uses vertex 0. Were the powergraph reference implementations updated to also use vertex 0 when this change was made to the MPGraph library?)

• We need to provide a powergraph implementation for connected components that can be compared with the MPGraph implementation.

• Can we provide a BC implementation for powergraph for correctness testing (probably)?

• How we will provide correctness testing for the sampleBC implementation when sampling is used? Also, note that the naming convention for "simpleXXX" is built into the regressions/Makefile. This means that the naming "sampleBC" can not be used with the regressions/Makefile unless it is renamed.

• We should be collecting statistics on CPU and GPU utilization and memory bandwidth. I do not believe that we need to instrument IO (while the tests read large graphs, the timing data is only for the actual graph primitive under test, not the IO).

[1] https://sourceforge.net/p/mpgraph/tickets/8/ (Regression testing for different implementations)

[Bryan Thompson]

• Description has changed:

Diff:

--- old
+++ new
@@ -1 +1,13 @@
-The existing performance and correctness regression test suite needs to be updated with the additional algorithms that are also implemented by/in powergraph, especially CC and BC if that is viable. 
+The performance and correctness regressions/ directory needs to be updated to capture (a) all algorithms and (b) all implementations (CUDA, OpenMP, TBB) [1].
+
+- Powergraph is difficult to install.  We need to setup a machine on which powergraph is correctly installed for regression testing in CI.
+
+- We need to verify that the regression tests run correctly for the different mpgraph algorithms when compared to powergraph.  (As part of this, verify that the same starting vertex is used for the powergraph and mpgraph implementations. The original code used the vertex with the highest degree, but the current code uses vertex 0.  Were the powergraph reference implementations updated to also use vertex 0 when this change was made to the MPGraph library?)
+
+- We need to provide a powergraph implementation for connected components that can be compared with the MPGraph implementation.
+
+- Can we provide a BC implementation for powergraph for correctness testing (probably)?  
+
+- How we will provide correctness testing for the sampleBC implementation when sampling is used?  Also, note that the naming convention for "simpleXXX" is built into the regressions/Makefile.  This means that the naming "sampleBC" can not be used with the regressions/Makefile unless it is renamed.
+
+[1] https://sourceforge.net/p/mpgraph/tickets/8/ (Regression testing for different implementations)

[Bryan Thompson]

Updated the ticket description for the regression test suite.

[Bryan Thompson]

• Description has changed:

Diff:

--- old
+++ new
@@ -10,4 +10,6 @@


 - How we will provide correctness testing for the sampleBC implementation when sampling is used?  Also, note that the naming convention for "simpleXXX" is built into the regressions/Makefile.  This means that the naming "sampleBC" can not be used with the regressions/Makefile unless it is renamed.

+- We should be collecting statistics on CPU and GPU utilization and memory bandwidth.  I do not believe that we need to instrument IO (while the tests read large graphs, the timing data is only for the actual graph primitive under test, not the IO).
+
 [1] https://sourceforge.net/p/mpgraph/tickets/8/ (Regression testing for different implementations)