Create a shim with rustup

[MaikKlein]

Currently rlsl needs a custom compiler but almost all changes have been merged into rustc. I think we should move away from building rustc ourselves and rely on a specific nightly version.

rlsl than just calls

rustup run nighty-version rlsl_shim

The specific nightly version will be managed behind the scenes.

The benefit is that the user doesn't have to build rustc manually. And rlsl should be easily installable. The only thing that has to be managed is the custom standard library.

[mitchmindtree]

Hi @MaikKlein, I'm currently having a go at understanding the rlsl repo while attempting to make it a little easier to contribute and get started. This issue looks like a good place to start!

I'd like to have a go at this, but have a couple questions:

If most of the necessary changes have been merged into rustc, why is it that we need a rustup shim? Is it so that it makes adding the custom standard library easier? Or is there something else that a rustup shim would provide over just using the latest nightly?

[MaikKlein]

If most of the necessary changes have been merged into rustc, why is it that we need a rustup shim? Is it so that it makes adding the custom standard library easier? Or is there something else that a rustup shim would provide over just using the latest nightly?

rustc is not statically linked. At the moment a custom version of rustc is compiled and put under $HOME/.rlsl. But I would like to move away from that and rely on a specific nightly version. We still need to compile the custom libstd.

But some changes still need to be made. For example I would like access to mir graphviz which is currently mostly private at the moment. I also have a incoming change for a more configurable inlining. And the biggest part is to re implement RustcDefaultCalls::list_metadata.

rlsl currently still relies on a public RustcDefaultCalls::list_metadata, a lot of stuff inside this function is private. I just haven't had the time yet to look at it.

Right now this is probably not the best issue to contribute too. I am currently working on a rendering framework, manually writing vulkan was just too hard to maintain. This took me way longer than I anticipated but I am almost finished with what I need. Now I am going to rewrite the testing infrastructure, so that we can easily add more tests, and then I am going to rewrite a big part of the compiler and simultaneously document everything. I also haven't yet created the CI pipeline.

I also can't accept any major contributions yet because it is my final university project. If you really want to help out now, I guess you could wrap #58 https://github.com/KhronosGroup/SPIRV-Tools, especially spirv-val and spirv-opt as a library would be extremely useful.

Otherwise you could address a few issues in the rust compiler like #45

https://github.com/MaikKlein/rlsl/blob/master/rlsl/src/lib.rs#L885-L894 running the inliner only works if the items is annotated with inline, it would be nice to force inlining. This is important for things like Deref, where the function always returns a reference.

If anything is unclear, feel free to ask.

[mitchmindtree]

Thanks a lot for the update!

I'll have a look into the feasibility of binding to SPIRV-Tools, though from my understanding rust-bindgen is still quite flakey/limited in terms of it's ability to bind to some parts of C++. It seems that rspirv is a pure-rust re-write of parts of SPIRV-Tools, but it's not yet clear to me how much it actually encompasses and whether or not it includes (or plans to include) spirv-val or spirv-opt. It looks like you're already using it in some places within rlsl so I'm guessing it does not fit the bill?

Best of luck with your university work!

[GabrielMajeri]

@mitchmindtree rspirv is awesome. It currently only has the common library parts from spirv-tools, they didn't port the validator and / or optimizer yet. The maintainer is really invested in the Rust / Vulkan ecosystem, and I'm sure they'll appreciate contributions (although it might be better to keep the optimizer / validator in a separate repository, and use the crate's public API for data representation).

That being said, if you do want to wrap spirv-opt or spirv-val, it might be a better idea to write a C++-to-C wrapper first, then generate bindings for that. The shaderc crate uses the C bindings to the C++ shaderc wrapper for spirv-tools and glslangValidator.

As MaikKlein said, we can do a lot to contribute to Rust's (graphics) ecosystem, until rspirv matures.

[mitchmindtree]

@GabrielMajeri thanks for the insight! I'm with you in that I'd much prefer to see a pure-rust alternative by supporting the rspirv project (e.g. rspirv-val and rspirv-opt crates). I'll have a look at the scale of what's involved first before deciding which road to go down. Perhaps bindings to spirv-tools could make a relatively quick-and-easy temporary solution.

[MaikKlein]

@mitchmindtree Both projects would probably be a huge undertaking. spirv-val is not feature complete and it has been around for a while, and I wouldn't underestimate the spir-v spec.

For spir-v opt, it would probably be better to use something like llvm, unless you really want to write those optimization passes yourself (which is totally fine). I am not sure if someone is actively working on a true bidirectional spirv llvmir compiler. The ones that I have seen only support "OpenCL" spir-v.

[dvc94ch]

Can't rlsl be implemented as a rustc plugin, until proc-macro-2 has more features (stable diagnostics api, some kind of support for whole crate analysis)?