Branch for functional testing design

functional-tests/README.md

@@ -0,0 +1,13 @@
+# Functional testing
+
+This directory contains the functional tests.
+
+Unlike the unit tests, these tests are _not_ part of the `pints` module, and
+they won't be installed via pypi/pip.
+
+I even imagine this directory (or the results subdirectory?) might be a
+submodule, so that test results etc. live in their own repository.
+
+
+
+

functional-tests/ft_haario_bardenet_mcmc_banana.py

@@ -0,0 +1,32 @@
+#!/usr/bin/env python3
+#
+# Functional test for Haario-Bardenet MCMC on the Banana problem
+#
+# unittest style
+#
+# This file is part of PINTS (https://github.com/pints-team/pints/) which is
+# released under the BSD 3-clause license. See accompanying LICENSE.md for
+# copyright notice and full license details.
+#
+import functional_testing
+import pints
+
+
+class HaarioBardenetMCMCBananaTest(functional_testing.Test):
+    # This test is discovered because it is in a file called ft_...
+    # It implements a shared interface, so that it can maybe call some methods
+    # to customise things, or to do a set-up and tear-down
+
+    def run(self):
+
+        problem = ...
+
+        controller = ...
+
+        run
+
+        return 1,2,3
+
+
+    def run_2(self):
+        # Maybe multiple tests can go in one file?

functional-tests/ft_haario_bardenet_mcmc_banana_2.py

@@ -0,0 +1,32 @@
+#!/usr/bin/env python3
+#
+# Functional test for Haario-Bardenet MCMC on the Banana problem
+#
+# PyTest style
+#
+# This file is part of PINTS (https://github.com/pints-team/pints/) which is
+# released under the BSD 3-clause license. See accompanying LICENSE.md for
+# copyright notice and full license details.
+#
+import functional_testing
+import pints
+
+
+def test_haario_bardenet_mcmc_banana(self):
+    # This test is discovered because it's in a directory ft_...
+    # And because its name starts with test_
+    # Arguments can be passed to functional testing by using a decorator, e.g.
+    #   @specialthing(1, 2, 3)
+    #   def test_haario_bardenet...
+
+    problem = ...
+
+    controller = ...
+
+    run
+
+    return 1,2,3
+
+
+def test_haario_norma(self):
+    # Again, we could have a method's tests grouped in a single file

functional-tests/results/some_result.cscv

@@ -0,0 +1,8 @@
+Hello!
+I am a CSV file
+
+Not sure about this bit yet.
+
+One per test? One per test group? One massive json file?
+
+How do we update this after running in actions? With git? With some sort of artefact thing?