Allow different Fortran source directory

[gold2718]

Add a new metadata table keyword, source_path, which allows a Fortran source file to be in a different directory from its metadata file. source_path is a path relative to the metadata file path.

User interface changes?: Yes
New optional metadata table keyword, source_path.
Backwards compatibility is maintained by using the metadata file path as the default value for the Fortran source path.

Testing:
test removed: none
unit tests: pass
system tests: pass (capgen test modified to test new feature)
manual testing: NA

Fixes #674
Fixes #676

[gold2718]

I have a couple of questions about this PR:

1. I'm not in love with the name, source_path. We now have relative_path (for dependencies) and source_path (for Fortran source files). Any ideas?
2. When a metadata file has multiple tables, I am currently only checking the first one. Do we want to require (and check) that all the tables have the same source_path? Since there is supposed to be a single Fortran file or the metadata file, I'm not sure what we want here.

[climbfuji]

I have a couple of questions about this PR:

1. I'm not in love with the name, source_path. We now have relative_path (for dependencies) and source_path (for Fortran source files). Any ideas?
2. When a metadata file has multiple tables, I am currently only checking the first one. Do we want to require (and check) that all the tables have the same source_path? Since there is supposed to be a single Fortran file or the metadata file, I'm not sure what we want here.

I also have a couple of questions!

Regarding your question 1. Would it be too difficult to use/reuse/abuse the existing relative_path and just search for the Fortran source file that corresponds to the .meta file? We know that the name must be the same, all we need to do is search all paths in relative_path (since this is a list) for thismetadatafilewithoutfileending.fortranfileendigs. This would be a "new-in-capgen" thing, since we cannot break the assumption that Fortran and Metadata file are in separate directories in prebuild.

Regarding your question 2. How is capgen handling the relative_path entires for the multiple tables?

In general, the use case for this new functionality is to be able to add metadata files for code that we do not "own", e.g. when we pull in a physics scheme for an authoritative source as a submodule?

[gold2718]

I thought relative_path was for dependencies, are you sure it would not cause any issues to use it for the source file location? In particular, I'm worried about a case where there might be more than one file with the correct name. If you discount this, I could try to adapt my PR to do the search (and remove the new keyword).

[climbfuji]

I thought relative_path was for dependencies, are you sure it would not cause any issues to use it for the source file location? In particular, I'm worried about a case where there might be more than one file with the correct name. If you discount this, I could try to adapt my PR to do the search (and remove the new keyword).

I am fairly confident that there is currently no case where there is more than one file with the correct name, and it wouldn't be overly restrictive to make this a requirement?

[gold2718]

I am fairly confident that there is currently no case where there is more than one file with the correct name, and it wouldn't be overly restrictive to make this a requirement?

I've seen duplicate filenames in a number of projects. DART comes to mind. RRTMGP has two files named mo_gas_optics_rrtmgp_kernels.F90 (and possibly other duplicates, I did not check).

[climbfuji]

I am fairly confident that there is currently no case where there is more than one file with the correct name, and it wouldn't be overly restrictive to make this a requirement?

I've seen duplicate filenames in a number of projects. DART comes to mind. RRTMGP has two files named mo_gas_optics_rrtmgp_kernels.F90 (and possibly other duplicates, I did not check).

True, but if I understand this PR and the underlying issue correctly, the restrictions would apply to files that correspond to a CCPP scheme (i.e. a metadata file with a certain prefix) only? In other words, if mp_thompson.meta was looking for a Fortran file that contains the CCPP entry points for the Thompson microphysics scheme, we would require that there can only be one mp_thompson.{F90,F,f,f90} in any of the directories that mp_thompson.meta lists in relative_path?

[peverwhee]

@gold2718 @climbfuji

I'd vote for a separate (optional) metadata field as Steve implemented here, rather than extending the use of relative_path. It feels clearer and more explicit (i.e. the user has to specify explicitly where the source file is, which would hopefully encourage people to keep the metadata and source files together).

As for the name, I can't think of anything better than source_path myself. In a perfect world, I'd say we should rename relative_path, but I don't want to break existing, functioning metadata files.

[climbfuji]

relative_path is all over the place, but it can be changed easily witha recursive search and replace call. I'd rather rename relative_path to dependency_path or dependencies_path or something else that makes more sense when paired with source_path.

[peverwhee]

one optional thing

[peverwhee]

thanks for fixing this!

[peverwhee]

oops! I know this is unrelated to your changes, but can you remove this residual dynamic constituents line left by me?

[dustinswales]

Looks good to me.
Thanks for adding this functionality. It will come in handy for rte-rrtmgp!