Code: add validate_remote_exec_path method to check executable (#5184)

A common problem is that the filepath of the executable for remote codes
is mistyped by accident. The user often doesn't realize until they
launch a calculation and it mysteriously fails with a non-descript
error. They have to look into the output files to find that the
executable could not be found.

At that point, it is not trivial to correct the mistake because the
`Code` cannot be edited nor can it be deleted, without first deleting
the calculation that was just run first. Therefore, it would be nice to
warn the user at the time of the code creation or storing.

However, the check requires opening a connection to the associated
computer which carries both significant overhead, and it may not always
be available at time of the code creation. Setup scripts for automated
environments may want to configure the computers and codes at a time
when they cannot be necessarily reached. Therefore, preventing codes
from being created in this case is not acceptable.

The compromise is to implement the check in `validate_remote_exec_path`
which can then freely be called by a user to check if the executable of
the remote code is usable. The method is added to the CLI through the
addition of the command `verdi code test`. Also here, we decide to not
add the check by default to `verdi code setup` as that should be able to
function without internet connection and with minimal overhead. The docs
are updated to encourage the user to run `verdi code test` before using
it in any calculations if they want to make sure it is functioning. In
the future, additional checks can be added to this command.

Author: sphuber

aiida/cmdline/commands/cmd_code.py

@@ -18,6 +18,7 @@
 from aiida.cmdline.params.options.commands import code as options_code
 from aiida.cmdline.utils import echo
 from aiida.cmdline.utils.decorators import with_dbenv
+from aiida.common import exceptions
 
 
 @verdi.group('code')
@@ -104,6 +105,26 @@ def setup_code(ctx, non_interactive, **kwargs):
     echo.echo_success(f'Code<{code.pk}> {code.full_label} created')
 
 
+@verdi_code.command('test')
+@arguments.CODE(callback=set_code_builder)
+@with_dbenv()
+def code_test(code):
+    """Run tests for the given code to check whether it is usable.
+
+    For remote codes the following checks are performed:
+
+     * Whether the remote executable exists.
+
+    """
+    if not code.is_local():
+        try:
+            code.validate_remote_exec_path()
+        except exceptions.ValidationError as exception:
+            echo.echo_critical(f'validation failed: {exception}')
+
+    echo.echo_success('all tests succeeded.')
+
+
 # Defining the ``COMPUTER`` option first guarantees that the user is prompted for the computer first. This is necessary
 # because the ``LABEL`` option has a callback that relies on the computer being already set. Execution order is
 # guaranteed only for the interactive case, however. For the non-interactive case, the callback is called explicitly

aiida/orm/nodes/data/code.py

@@ -11,6 +11,7 @@
 import os
 
 from aiida.common import exceptions
+from aiida.common.log import override_log_level
 
 from .data import Data
 
@@ -293,6 +294,32 @@ def _validate(self):
             if not self.get_remote_exec_path():
                 raise exceptions.ValidationError('You did not specify a remote executable')
 
+    def validate_remote_exec_path(self):
+        """Validate the ``remote_exec_path`` attribute.
+
+        Checks whether the executable exists on the remote computer if a transport can be opened to it. This method
+        is intentionally not called in ``_validate`` as to allow the creation of ``Code`` instances whose computers can
+        not yet be connected to and as to not require the overhead of opening transports in storing a new code.
+
+        :raises `~aiida.common.exceptions.ValidationError`: if no transport could be opened or if the defined executable
+            does not exist on the remote computer.
+        """
+        filepath = self.get_remote_exec_path()
+
+        try:
+            with override_log_level():  # Temporarily suppress noisy logging
+                with self.computer.get_transport() as transport:
+                    file_exists = transport.isfile(filepath)
+        except Exception:  # pylint: disable=broad-except
+            raise exceptions.ValidationError(
+                'Could not connect to the configured computer to determine whether the specified executable exists.'
+            )
+
+        if not file_exists:
+            raise exceptions.ValidationError(
+                f'the provided remote absolute path `{filepath}` does not exist on the computer.'
+            )
+
     def set_prepend_text(self, code):
         """
         Pass a string of code that will be put in the scheduler script before the

docs/source/howto/run_codes.rst

@@ -265,6 +265,13 @@ Use this, for instance, to load modules or set variables that are needed by the
 
 At the end, you receive a confirmation, with the *PK* and the *UUID* of your new code.
 
+.. tip::
+
+    The ``verdi code setup`` command performs minimal checks in order to keep it performant and not rely on an internet connection.
+    If you want additional checks to verify the code is properly configured and usable, run the `verdi code test` command.
+    For remote codes for example, this will check whether the associated computer can be connected to and whether the specified executable exists.
+    Look at the command help to see what other checks may be run.
+
 .. admonition:: Using configuration files
     :class: tip title-icon-lightbulb
 

docs/source/reference/command_line.rst

@@ -78,6 +78,7 @@ Below is a list with all available subcommands.
       reveal     Reveal one or more hidden codes in `verdi code list`.
       setup      Setup a new code.
       show       Display detailed information for a code.
+      test       Run tests for the given code to check whether it is usable.
 
 
 .. _reference:command-line:verdi-computer:

tests/cmdline/commands/test_code.py

@@ -19,7 +19,7 @@
 from aiida.cmdline.commands import cmd_code
 from aiida.cmdline.params.options.commands.code import validate_label_uniqueness
 from aiida.common.exceptions import MultipleObjectsError, NotExistent
-from aiida.orm import Code, load_code
+from aiida.orm import Code, Computer, load_code
 
 
 @pytest.fixture
@@ -303,7 +303,7 @@ def test_code_setup_remote_duplicate_full_label_non_interactive(
     run_cli_command, aiida_local_code_factory, aiida_localhost, label_first
 ):
     """Test ``verdi code setup`` for a remote code in non-interactive mode specifying an existing full label."""
-    label = 'some-label'
+    label = f'some-label-{label_first}'
     aiida_local_code_factory('core.arithmetic.add', '/bin/cat', computer=aiida_localhost, label=label)
     assert isinstance(load_code(label), Code)
 
@@ -318,7 +318,6 @@ def test_code_setup_remote_duplicate_full_label_non_interactive(
     assert f'the code `{label}@{aiida_localhost.label}` already exists.' in result.output
 
 
-@pytest.mark.usefixtures('clear_database_before_test')
 @pytest.mark.parametrize('non_interactive_editor', ('sleep 1; vim -cwq',), indirect=True)
 def test_code_setup_local_duplicate_full_label_interactive(
     run_cli_command, aiida_local_code_factory, aiida_localhost, non_interactive_editor
@@ -377,3 +376,24 @@ def load_code(*args, **kwargs):
 
     with pytest.raises(click.BadParameter, match=r'multiple copies of the local code `.*` already exist.'):
         validate_label_uniqueness(ctx, None, 'some-code')
+
+
+@pytest.mark.usefixtures('clear_database_before_test')
+def test_code_test(run_cli_command):
+    """Test the ``verdi code test`` command."""
+    computer = Computer(
+        label='test-code-computer', transport_type='core.local', hostname='localhost', scheduler_type='core.slurm'
+    ).store()
+    code = Code(remote_computer_exec=[computer, '/bin/invalid']).store()
+
+    result = run_cli_command(cmd_code.code_test, [str(code.pk)], raises=True)
+    assert 'Could not connect to the configured computer' in result.output
+
+    computer.configure()
+
+    result = run_cli_command(cmd_code.code_test, [str(code.pk)], raises=True)
+    assert 'the provided remote absolute path `/bin/invalid` does not exist' in result.output
+
+    code = Code(remote_computer_exec=[computer, '/bin/bash']).store()
+    result = run_cli_command(cmd_code.code_test, [str(code.pk)])
+    assert 'all tests succeeded.' in result.output

tests/orm/data/test_code.py

@@ -0,0 +1,35 @@
+# -*- coding: utf-8 -*-
+###########################################################################
+# Copyright (c), The AiiDA team. All rights reserved.                     #
+# This file is part of the AiiDA code.                                    #
+#                                                                         #
+# The code is hosted on GitHub at https://github.com/aiidateam/aiida-core #
+# For further information on the license, see the LICENSE.txt file        #
+# For further information please visit http://www.aiida.net               #
+###########################################################################
+# pylint: disable=redefined-outer-name
+"""Tests for :class:`aiida.orm.nodes.data.code.Code` class."""
+import pytest
+
+from aiida.common.exceptions import ValidationError
+from aiida.orm import Code, Computer
+
+
+@pytest.mark.usefixtures('clear_database_before_test')
+def test_validate_remote_exec_path():
+    """Test ``Code.validate_remote_exec_path``."""
+    computer = Computer(
+        label='test-code-computer', transport_type='core.local', hostname='localhost', scheduler_type='core.slurm'
+    ).store()
+    code = Code(remote_computer_exec=[computer, '/bin/invalid'])
+
+    with pytest.raises(ValidationError, match=r'Could not connect to the configured computer.*'):
+        code.validate_remote_exec_path()
+
+    computer.configure()
+
+    with pytest.raises(ValidationError, match=r'the provided remote absolute path `.*` does not exist.*'):
+        code.validate_remote_exec_path()
+
+    code = Code(remote_computer_exec=[computer, '/bin/bash'])
+    code.validate_remote_exec_path()