Bug: Files don't get updated on exercism download ... if submitted

[sshine]

The main purpose of this issue is to document a usability bug caused by the way exercism download ... currently updates exercises on the Haskell track. It seems that there are some situations where exercise version migrations cause problems, and this issue describes three such edge cases.

A similar but not equivalent problem is experienced exercism/ocaml#368.

If users return to an exercise that has been updated in the meantime, and they click the website button that forwards their solution to the latest version, and they type exercism download --exercise=... --track=haskell, all the files that were updated are downloaded, except the ones the student once ran exercism submit ... on.

This causes the following problems on the Haskell track:

1. The solution file itself cannot be overriden by a stub, since that would mean that the student loses their solution. But if the stub has breaking changes, such as changes to a type signature, or an exported data type, the old solution may no longer successfully compile against test/Tests.hs, and there may not be any clear hint on how to proceed: While the stub file does contain the most recent type signatures or data types, they are often deliberately inferred in the test suite -- for example to allow for overloaded solutions. So the student has a solution that won't compile and no clear indication of what is missing to make it compile.

2. The project file, package.yaml may have been submitted so that the solution can compile on the mentor's or the automated mentoring system's machine. So if the student submitted a package.yaml, and a later update to the exercise depends on packages that are neither listed in the previous version or the student version of package.yaml, the solution may no longer successfully compile against test/Tests.hs, and the error messages will report of modules that don't exist, but not necessarily suggest what packages should be added to make these modules available.

3. The test file, test/Tests.hs may have been submitted because the student has enabled a commented-out test, or supplied a test of their own, to show the mentor that it passes this. For example, the Diamond exercise received a property-based test suite in Diamond: Implement property-based tests #843 that would seem ideal for the student to improve on, since the README essentially contains recipes for more properties. But if the student submits improved properties, they will never receive updates to the ones handed out.

[petertseng]

Investigate whether exercism/website#2049 helps with this. Supposedly now all editor.files are included in the important files which calculate whether an exercise has changed.