Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cross Compilation: add a sv2-rpi.sh script to generate tarball distributions for Raspberry Pi OS #684

Closed
wants to merge 15 commits into from

Conversation

plebhash
Copy link
Collaborator

@plebhash plebhash commented Nov 27, 2023

Most edge network hardware that will run the SRI stack will most likely not be x86-64 cloud servers.

A much more likely target will be ARM SoCs and SBCs, especially when it comes to pleb mining.

Therefore it is important to make sure SRI supports the most popular off-the-shelf platforms on this category (e.g.: Raspberry Pi).

This contribution is also the first step towards SRI supporting ASIC Control Boards in the future.

stratum-rpi

This PR introduces a shell script (sri-rpi.sh) that automates cross compilation for tarball distributions (32 and 64 bits) of the different SRI executables.

sv2-rpi.sh

Here is how to use sv2-rpi.sh:

Usage:
$ ./sv2-rpi.sh <role> <bits>

Available options for <role> are:
jd_client
jd_server
mining_proxy_sv2
pool_sv2
translator_sv2

Available options for <bits> are:
32
64

For example:
$ ./sv2-rpi.sh jd_server 64

After the package generation, you are ready to scp the tarball to the RPi:

$  scp tar/jd_server-aarch64-unknown-linux-gnu.tar.gz <rpiuser>@<your.rpi.ip.addr>:/home/<rpiuser>

package layout

Configuration files are placed under /etc/sri, and executables are placed under /bin. For example, here is the tarball layout of jd_server:

├── bin
│   └── jd_server
└── etc
    └── sri
        ├── jds-config-hosted-example.toml
        └── jds-config-local-example.toml

cargo-cross

Under the hood, sv2-rpi.sh leverages cargo-cross, so it is a pre-requisite for executing the script. It can be installed via

$ cargo install cross

Toolchain management via cargo-cross enables a stable cross-compilation experience. It also requires that at least one of these dependencies is available:

sv2-rpi.sh Outdated Show resolved Hide resolved
sv2-rpi.sh Outdated Show resolved Hide resolved
@plebhash plebhash marked this pull request as ready for review December 28, 2023 20:11
@s2dd
Copy link

s2dd commented Dec 29, 2023

just some alt. variant = i can compile sv2 roles direct on a rpi and get a bin for dist.

cross compile for Antminer hosting tProxy direct is possible, a Antminer CB(Controlboard) has plenty CPU and Mem left when running its FW, a cross compile of tProxy, then can scp to device and set regular FW UI pool to 127.0.0.1:port that in turn points to next ie. pool-endpoint ...

this for ppl with one or few machines not wanting to run role on a separate machine ...

@GitGab19
Copy link
Collaborator

GitGab19 commented Jan 3, 2024

@plebhash let us know if you think this PR is completely ready.
I just saw you pushed some changes recently, and also added cargo cross as suggested by @Fi3

@Fi3
Copy link
Collaborator

Fi3 commented Jan 3, 2024

@GitGab19 yesterday I suggested to add a github action that check if cross compilation works: just try to compile with cargo cross from a linux vm for the target that we care about and check that no errors or warning are emitted.

@GitGab19
Copy link
Collaborator

GitGab19 commented Jan 3, 2024

@GitGab19 yesterday I suggested to add a github action that check if cross compilation works: just try to compile with cargo cross from a linux vm for the target that we care about and check that no errors or warning are emitted.

Right, thanks for better explanation (I failed in taking notes on that yesterday 😆).
@plebhash so please have a look at @Fi3's suggestion for this.

@plebhash
Copy link
Collaborator Author

plebhash commented Jan 4, 2024

just some alt. variant = i can compile sv2 roles direct on a rpi and get a bin for dist.

cross compile for Antminer hosting tProxy direct is possible, a Antminer CB(Controlboard) has plenty CPU and Mem left when running its FW, a cross compile of tProxy, then can scp to device and set regular FW UI pool to 127.0.0.1:port that in turn points to next ie. pool-endpoint ...

this for ppl with one or few machines not wanting to run role on a separate machine ...

thanks for your input @s2dd
indeed, this script can be seen as redundant or overkill, depending on the perspective

however, the vision here is to have a way to aid non-technical users to deploy SV2/SRI infrastructure with the smallest possible friction (and avoid long compilation times natively on the RPi)

once SRI has stable releases, we can simply add the aarch64-unknown-linux-gnu and arm-unknown-linux-gnueabi binaries as part of the official distribution. Then, users can simply download those binaries and get them running out-of-the-box on their RPis without the need to install rust toolchain etc.

end of the day, the goal is to boost SV2 adoption by combining these two factors:

  • pleb-friendly hardware
  • low friction UX

@s2dd
Copy link

s2dd commented Jan 5, 2024

yes but to get a rpi dist it could be compiled on rpi was what i tried convey

also i find cross-compile tProxy for Antminer a interesting path forward

@plebhash
Copy link
Collaborator Author

plebhash commented Jan 5, 2024

yes but to get a rpi dist it could be compiled on rpi was what i tried convey

compiling natively on the RPi is not ideal for a release workflow for multiple reasons:

  • it is much slower, since the RPi SoC has very limited computation power
  • the team would need to maintain two physical RPis (32+64 bits) dedicated for this purpose

during my experience in the Embedded Software industry I learned that cross compilation workflows should always be preferred for production scenarios, while native compilation should only be a last resort for quick-n-dirty PoCs

also i find cross-compile tProxy for Antminer a interesting path forward

that is a good idea, I wrote this issue to keep track of it #711

@plebhash
Copy link
Collaborator Author

plebhash commented Jan 5, 2024

@GitGab19 yesterday I suggested to add a github action that check if cross compilation works: just try to compile with cargo cross from a linux vm for the target that we care about and check that no errors or warning are emitted.

done

the workflow calls sv2-rpi.sh for all roles on both 32 and 64 bits, and then checks if the binary artifacts exists and the tarballs were created correctly

this is not necessarily checking for cross compilation warnings or errors like @Fi3 suggested, since cargo cross is actually invoked by sv2-rpi.sh.
if cross-compilation fails, the build artifacts will not be present, and the Check artifacts step will fail

if we eventually decide to expand cross-compilation support to different targets beyond only aarch64-unknown-linux-gnu + arm-unknown-linux-gnueabi (RPi 64 and 32 bits, respectively), it might make sense to add a new workflow that uses some Github Action specifically tailored for this task (e.g.: houseabsolute/actions-rust-cross@v0)... that would allow for more detailed control over the different aspects of cross-compilation and what could be going wrong for different achitectures

however that is probably overkill at the moment, IMO

@plebhash
Copy link
Collaborator Author

closing in favor of #815

@plebhash plebhash closed this Mar 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants