#[diplomat::include] performance improvements

[tyler-zeromatter]

I don't think this at present warrants an issue, and with all the discussion around platforms for community discussion I thought this would be a good place to post.

Been thinking about @Manishearth's performance concerns around #[diplomat::include]. I think any performance improvement to #[diplomat::include] as it is used by the proc macro is (at present) not necessary, but I just want to throw out some ideas that we can revisit if performance ever becomes an issue.

[tyler-zeromatter]

Since Diplomat's proc macros are each evaluated separately, one solution that comes to mind is a TCP server that can evaluate requests sent by each proc_macro instance to return syn structs as raw memory. I think the main thing to research would be a way to spin up a server in concert with the diplomat macro so that we have a guarantee that the memory layout of the syn structures is the same between processes.

I'm not sure how a server could slot neatly into the proc_macro build pipeline, and this is absolutely overkill for the way #[diplomat::include] is used at present, but I at least want this out there as something we could revisit.

[Manishearth]

I think this would be too heavy and people would be concerned about TCP during the build process. It's a neat idea though.

[tyler-zeromatter]

In a similar vein, we could have a file cache with the syn bytedata, and some versioning information to ensure the accuracy of the cache. It all depends on if the read from disk of the cache is faster than compiling the syn information. Currently, I would say not

[Manishearth]

Yeah, I mostly want us to measure performance here and see if we should be encouraging designs that use include or not.

[tyler-zeromatter]

I'll reiterate my notes from the lightweight generics issue:

We don't have a lot of definitions, but because there's no caching in the macro, parsing the syn info of each #[diplomat::include] and adding it to the module takes around 4-6ms. That's around 55ms total of parsing #[diplomat::include] statements for the whole project. It could get bigger if we ever implement more macro features like recursion.

I think it should still remain pretty small for traits, if the syntax is limited to defining an empty trait and what types it's implemented for; not sure if we're interested in expanding the syntax for more features?

[Manishearth]

Yeah, I went through my notifications in the wrong order.

[tyler-zeromatter]

No worries! I'm hopping all over the place right now, discussion is very active today