Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

rust: Replace keep_fingerprints logic with unconditional cargo clean #63

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

rust: Replace keep_fingerprints logic with unconditional cargo clean #63

wants to merge 3 commits into from

Commits on Aug 19, 2024

  1. rust: Avoid rename-output.sh output if there is no output 

    This avoids producing:
```
+lint | no files found within ./target matching the provided output regexp
+lint | find: '/tmp/earthly/lib/rust': No such file or directory
```
when there is no output, because `copy-output.sh` only creates that path `if [
-n \"\$1\" ]`.

Apply the same condition to the `rename-output.sh` end of things.
    @ijc
    ijc committed Aug 19, 2024
    Configuration menu
    Copy the full SHA
    b8f3ea5 View commit details
    Browse the repository at this point in the history

  2. rust: Echo cargo $args into the logs 

    The logs otherwise just contain:
```
--> RUN set -e; cargo $args; cargo sweep -r -t $EARTHLY_SWEEP_DAYS; cargo sweep -r -i; $EARTHLY_FUNCTIONS_HOME/copy-output.sh "$output";
```

which doesn't show what is actually being run.
    @ijc
    ijc committed Aug 19, 2024
    Configuration menu
    Copy the full SHA
    a9b9b6e View commit details
    Browse the repository at this point in the history

  3. rust: Replace keep_fingerprints logic with unconditional cargo clean 

    Due to Earthly's layer cache code added with `COPY` (even with `--keep-ts`) can
end up with timestamps (`mtime`) corresponding to the point of creation of the
cache entry, not the current time.

However on a following build the `target` mount cache may contain builds from
other branches, with different code for those dependencies, which have a newer
`mtime`. In this case `cargo` will think it can use the cached dependency
instead of rebuilding (because the code appears older than the cached entry
under `target`).

Avoid this by using `cargo clean` to remove the build artifacts for any local
crate. This should become unnecessary with rust-lang/cargo#14136

This replaces the old behaviour of removing the fingerprints directory. Using
`cargo clean` uses a proper cargo API rather than relying on implementation
details like where the fingerprints live and what the consequence removing them
is. It may also keep the cached data smaller since it removes the build
artifacts which will likely never be reused due to the lack of fingerprint.

Note that the previous fingerprint cleaning was subject to a race where a
different parallel build could reintroduce some fingerprints between `DO
+REMOVE_SOURCE_FINGERPRINTS` and the `RUN ... cargo $args`. For that reason the
calls to `cargo clean` here are made within the same `RUN` command so that the
target cache remains locked.

By switching to `cargo metadata` the requirement for `tomljson` is removed.
    @ijc
    ijc committed Aug 19, 2024
    Configuration menu
    Copy the full SHA
    d24d25c View commit details
    Browse the repository at this point in the history