How to build and deploy a pip package with multiple different rust --features flags?

[LLukas22]

Hello,
I'm currently updating my python bindings for the rustformers\llm project.

Rustformers now supports GPU support via the cublas \ clblast and metal feature flags. I was woundering if it would be possible to build pip packages in a way that would allow installing gpu accelerated packages via pip's optional feature flag e.g. pip install llm-rs[cublas].

After some googling i found the package.metadata.maturin field for Cargo.toml but i don't understand how it influences the build process. Maybe im doing something wrong?

Currently the features section of my Cargo.toml looks like this:

[features]
cublas = ["llm/cublas", "llm-base/cublas"]
clblast = ["llm/clblast", "llm-base/clblast"]
metal = ["llm/metal", "llm-base/metal"]

[package.metadata.maturin]
features = ["cublas", "clblast", "metal"]

If i debug my package via maturin develop -r --features cublas the feature seams to be active.

[konstin]

pip extras can only really be used to install other packages, so you can't use them to build different feature sets. Unfortunately selecting different architectures at least somewhat automatically is still an unsolved problem (torch even uses different index servers for different cuda versions, jax also has index servers and extras).

What you could try is to add shim crate for each target architecture (e.g. a llm-rs-cublas crate and a llm-rs-metal crate) and a llm-rs packages that does nothing but dispatch to the right backend package given the extra the user selected.

[konstin]

if you don't want to ship multiple architectures in one package, i'm afraid you'll have to create one pypi package for each architecture. It's really not very clean but given that even torch and jax don't seem to have any better ideas i'm afraid i can only recommend that

[LLukas22]

@konstin Sorry to bother you again, but i'm currently trying to create multiple differently named wheels for my package but can't get it working the way i expected it to. I found the following issue #313 where you recommended to create "wrapper" crates. Are there some examples of this somewhere as i can't seam to figure it out or should i consider switching to setuptools-rust?

[konstin]

I'd imagine something like

.
├── actual-impl-lib
│   ├── Cargo.toml
│   └── src
│       ├── bindings.rs
│       ├── error.rs
│       ├── lib.rs
│       └── math.rs
└── shims
    ├── wrapper-cublas
    │   ├── Cargo.toml
    │   └── src
    │       └── lib.rs
    └── wrapper-metal
        ├── Cargo.toml
        └── src
            └── lib.rs

Each wrapper crate only depends on you're actual implementation with the relevant feature. It's main property is the pypi package name and which features it activates.

In the CI job for platform you call

maturin build --release -m shims/wrapper-cublas/Cargo.toml
maturin build --release -m shims/wrapper-metal/Cargo.toml

or the equivalent maturin-action.

[LLukas22]

Ok after some thinkering i couldn't get the wrapper crates to work. I solved my problem by introducing another build step, which patches the name field in the pyproject.toml file to a value configurable via the environment. This way i have no changes in my "cpu-only" CI pipeline but i can simply change the package name for the "cuda","opencl" and "metal" pipelines by adding another step into the pipeline.

[LLukas22]

Ok got it, so basically i have to create multiple CI-Pipelines that build the packages with the desired feature-flags and then upload them to different pypi projects? E.g. something similar to this?