Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pull resources from NAR directly #19

Open
4 tasks
Thesola10 opened this issue Feb 27, 2025 · 1 comment
Open
4 tasks

Pull resources from NAR directly #19

Thesola10 opened this issue Feb 27, 2025 · 1 comment
Labels
type: enhancement New feature or request

Comments

@Thesola10
Copy link
Collaborator

Thesola10 commented Feb 27, 2025

Right now the static-bins output just crossed 100MB uncompressed, which means it now has the honor of slamming into cachix/cachix#619. (UPDATE: they kindly pushed back the size limit cachix/cachix#619 (comment), but the fragility of Nixie's current approach is still evident)

We need a simpler, more Nix-y way to retrieve files from within the derivation. My idea now that #16 has been merged, is to try and write a streamed NAR parser in Amber, which would in turn provide exact offsets to download a file from within the NAR, using minimal bandwidth (and therefore memory) to perform the lookup in the first place.

The ideal implementation would be a Jura package, which implies a couple fixes to upstream projects first:

@Thesola10 Thesola10 added the type: enhancement New feature or request label Feb 27, 2025
@Thesola10
Copy link
Collaborator Author

Thesola10 commented Mar 1, 2025

Well this is annoying... Cachix only supports zstd and xz as compression methods. zstd is completely out of the question, and xz isn't much better (not preinstalled on macOS for instance).

I really don't want to give up on the Nix cache approach since it ensures any version of the script can handle its resources format, so I'll have to look into self-hosting a Harmonia cache or something.

I'll keep working on amber-nar in the mean time.

... then again the static xz executable is tiny... and the static-bins derivation is only needed on supported platforms...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant