Skip to content

ruby-modules: improve cross-compilation support#270242

Merged
Artturin merged 2 commits intoNixOS:stagingfrom
tie:ruby-cross
Jun 25, 2024
Merged

ruby-modules: improve cross-compilation support#270242
Artturin merged 2 commits intoNixOS:stagingfrom
tie:ruby-cross

Conversation

@tie
Copy link
Member

@tie tie commented Nov 26, 2023

Description of changes

This PR contains fixes for cross-compilation in Ruby infrastructure. In particular,

  • bundler{App,Env} were using host ruby to execute gen-bin-stubs.rb, same for rsync if copyGemFiles was set.
  • buildRubyGem output contained Ruby-generated binstubs that referenced ruby from nativeBuildInputs.

For example,

nix build --impure --expr 'let
  pkgs = import ./. {
    localSystem.system = "aarch64-linux";
    crossSystem.config = "x86_64-unknown-linux-gnu";
  };
in
pkgs.doing'

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 23.11 Release Notes (or backporting 23.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Priorities

Add a 👍 reaction to pull requests you find important.

@github-actions github-actions bot added the 6.topic: ruby A dynamic, open source programming language with a focus on simplicity and productivity. label Nov 26, 2023
@tie tie added the 6.topic: cross-compilation Building packages on a different platform than they will be used on label Nov 26, 2023
@ofborg ofborg bot added 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-darwin: 1001-2500 This PR causes many rebuilds on Darwin and should most likely target the staging branches. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 2501-5000 This PR causes many rebuilds on Linux and should target the staging branches. labels Nov 27, 2023
@tie tie marked this pull request as ready for review January 12, 2024 05:21
@ofborg ofborg bot requested a review from tylerjl March 26, 2024 14:13
@ofborg ofborg bot added 10.rebuild-linux: 1001-2500 This PR causes many rebuilds on Linux and should target the staging branches. and removed 10.rebuild-linux: 2501-5000 This PR causes many rebuilds on Linux and should target the staging branches. labels Mar 26, 2024
@tie tie force-pushed the ruby-cross branch 2 times, most recently from 14d83fd to 9572406 Compare May 3, 2024 10:40
@ofborg ofborg bot requested a review from Rucadi May 3, 2024 12:12
@tie tie requested a review from SuperSandro2000 May 28, 2024 07:02
@tie
Copy link
Member Author

tie commented May 30, 2024

I’m really confused here, is this PR somehow blocklisted? Is Ruby ecosystem so dead that we don’t review changes anymore (e.g. #258705 also took a long time to merge)? Am I the only one actually using it (given that nobody complained about this issue before)? Do I have to sell my soul to the devil to get this PR reviewed? It’s been 6 months (that’s a full NixOS release cycle) with zero feedback from maintainers and requested reviewers. 😢

@tie tie requested review from Artturin and wegank May 30, 2024 23:18
@Artturin Artturin self-assigned this May 31, 2024
@anthonyroussel
Copy link
Member

anthonyroussel commented May 31, 2024

@tie Hello!

Sorry for the late reply, and thank you for your work on improving cross-compilation support for Ruby packages 😄

Your changes looks good to me, but unfortunately I don't have a computer powerful enough to test your changes (cross-compilation + ~2500 rebuilds) and check that everything's OK on my end.

As your PR targets the staging branch, I think we can merge and check that the builds are OK on Hydra during the next staging-next cycle?

@tie
Copy link
Member Author

tie commented May 31, 2024

Thank you, yes, that sounds good.

@anthonyroussel
Copy link
Member

anthonyroussel commented May 31, 2024

@ofborg build pkgsCross.gnu64.bundler pkgsCross.gnu64.puppet-bolt

@tie
Copy link
Member Author

tie commented May 31, 2024

I’m not really sure if puppet-bolt is a good test case without #316023.

Edit: yes, it doesn’t build without this PR, but does with 9b73495 cherry-picked (pkgsCross.gnu64 from aarch64-linux).

@SuperSandro2000
Copy link
Member

SuperSandro2000 commented Jun 2, 2024

Those changes are pretty simple and just correct that we use the correct binaries in shebangs, nothing compiler/linker related. I think we can safely merge this.

@ofborg ofborg bot requested a review from anthonyroussel June 3, 2024 22:32
@Artturin Artturin merged commit 897e070 into NixOS:staging Jun 25, 2024
@github-actions
Copy link
Contributor

Successfully created backport PR for staging-24.05:

@trofi
Copy link
Contributor

trofi commented Jun 30, 2024

Bisect says 3097117 ruby-modules: improve cross-compilation support broek weechat build in staging:

$ nix build --no-link -f. weechat -L
...
weechat> [ 53%] Built target doc-quickstart-pl
weechat> [ 53%] Built target doc-quickstart-ru
weechat> No such file or directory - /nix/store/m15q42klamlbf7czyk1bscjfnwsjr2ay-ruby3.1-pygments.rb-2.4.0/lib/ruby/gems/3.1.0/gems/pygments.rb-2.4.0/lib/pygments/mentos.py
weechat>   Use --trace to show backtrace

@trofi
Copy link
Contributor

trofi commented Jun 30, 2024

Probably due to unpatched shebang in the store:

$ head -n1 /nix/store/m15q42klamlbf7czyk1bscjfnwsjr2ay-ruby3.1-pygments.rb-2.4.0/lib/ruby/gems/3.1.0/gems/pygments.rb-2.4.0/lib/pygments/mentos.py
#!/usr/bin/env python3

@tie tie deleted the ruby-cross branch June 30, 2024 14:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.topic: cross-compilation Building packages on a different platform than they will be used on 6.topic: ruby A dynamic, open source programming language with a focus on simplicity and productivity. 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-darwin: 1001-2500 This PR causes many rebuilds on Darwin and should most likely target the staging branches. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 1001-2500 This PR causes many rebuilds on Linux and should target the staging branches.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

Comments