Dealing with include_bytes in dependencies

[l-monninger]

I'm having some trouble modding the source filters to enable include_bytes macros in dependencies to compile. Consider the source filter and output below:

allFilter = path: _type: true;

  src = pkgs.lib.cleanSourceWith {
      src = craneLib.path ./..; # The original, unfiltered source
      filter = allFilter;
  };
  # ...
  
  fileSetForCrate = crate: pkgs.lib.fileset.toSource {
        root = ../.;
        fileset = pkgs.lib.fileset.unions [
            ../Cargo.toml
            ../Cargo.lock
            crate
            # I think something should go here, to include the `vendor-cargo-deps`, but I'm not sure what.
        ];
    };

error: couldn't read /nix/store/b84qsq6cr5flyzcyqjqajn14r6fhi5fd-vendor-cargo-deps/b5940b9c4f2d743a9ad4ce78173ebd3277af4ad2aa05c25e6f32bc8e76dbf9cb/move-prover-0.1.0/src/../../../../aptos-move/framework/src/aptos-natives.bpl: No such file or directory (os error 2)
       >    --> /nix/store/b84qsq6cr5flyzcyqjqajn14r6fhi5fd-vendor-cargo-deps/b5940b9c4f2d743a9ad4ce78173ebd3277af4ad2aa05c25e6f32bc8e76dbf9cb/move-prover-0.1.0/src/cli.rs:774:33
       >     |
       > 774 |                   template_bytes: include_bytes!(
       >     |  _________________________________^
       > 775 | |                     "../../../../aptos-move/framework/src/aptos-natives.bpl"
       > 776 | |                 )
       >     | |_________________^
       >     |

Basically, it appears that pattern matching against vendor-cargo-deps does not include the file. This could be because of the pattern matching, but it could also be because of the filtering out crates not included in the Cargo.lock which takes place in vendorGitDeps.

Any ideas on how to solve this?

[ipetkov]

Crates are not supposed to depend on/include any files outside of their own directory. This happens to work out for git dependencies because cargo happens to check out the entire repo, but it would not work properly when the crate is published to a registry like crates.io (in which case cargo will only upload the crate's directory and not include any parent files).

There are two basic options to handle this:

1. Submit a fix for this upstream to not depend on any parent files.
2. Patch up the sources before they are vendored into the rest of the derivation: https://crane.dev/patching_dependency_sources.html

[l-monninger]

Thank you!