Unable to build with haero and in-tree CMake based EKAT build

[cameronrutherford]

See #64 for the context where the first failure was noticed.

HAERO's exported CMake configuration installs an nvcc wrapper into install/bin for ease of use.

However, when performing an in-tree CMake build of EKAT, the CMAKE_CXX_COMPILER ends up pointing to the nvcc_wrapper in the HAERO source directory, rather than the one that is installed in install/bin.

This means that HAERO is dependent on it's own source directory when installing, and to resolve this issue, HAERO installations should be independent of the resource code.

To observe this issue, simply install HAERO per default mam4xx configuration using the GPU, delete the cloned HAERO source code, and then attempt to install mam4xx. You will see an error that no CMAKE_CXX_COMPILER can be found, where the error has the path where the HAERO source director was.

@jeff-cohere also witnessed this issue.

[jeff-cohere]

The real solution in my opinion is to convince the EAMxx folks not to use in-tree builds for things like EKAT, because CMake build phases work in a way that makes these in-tree builds awkward. CMake's ExternalProject machinery is a far better way to handle third-party library builds. I don't think we can impose this on the EAMxx project at this point, so we might have to find a workaround for now.

[cameronrutherford]

The current workaround is to not delete the source files in CI, or when developing locally.

I agree that truly resolving this and not implementing just a workaround requires EAMMxx buy in