-
Notifications
You must be signed in to change notification settings - Fork 9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: vyper 0.4 support #110
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this checked against 0.3.10 and 0.4.0?
Sorry meant to make this a draft.. The tests are all pre-0.4.0, but I need to add more tests for 0.4.0 range. |
i think rather than using absolute paths, you can add the |
I tried this but was running into other issues, I can keep trying though but I was in a tangled web of confusion. |
Looks like vyper in the 0.3.0 range does not like the absolute paths |
WIP - still in development, please be patient. |
FYI @fubuloubu, it doesn't work yet on snekmate: https://github.com/pcaversaccio/snekmate/actions/runs/8651807519/job/23771914522#step:8:1 |
i think constructing the input dict from just a directory is going to be pretty complicated, because it's hard to predict how the compiler will interact with search path (cf. for instance: vyperlang/vyper@b0ea5b6, vyperlang/vyper@d372378, vyperlang/vyper@c6f457a, vyperlang/vyper@a3bc3eb, vyperlang/vyper@5d10ea0). i'm not sure what direction ape wants to take here, but maybe the best thing might be to call the compiler and recursively get its ast output (per vyperlang/vyper@9428196), or just use the cli and get |
Having the import graph would be extremely useful, in both directions (know which code modules import which other code modules so that a downstream module change will trigger us to recompile one that hasn't because of the dependency) |
@fubuloubu also check maybe the best thing is to ditch standard-json input here? because it's not trivial to construct. from CLI, everything just works since the compiler will resolve things in the filesystem automatically. if you want to keep using standard-json and one-shot the compilation (i.e., not have to query the compiler multiple times about a file), the other way would be to bundle everything ape knows is in the environment (hypothetical example: shove contracts/, libraries/, modules/) all into json input. that provides the best chance of successful compilation, although you still might end up getting divergences between what the compiler thinks is in the environment and what ape thinks is in the environment. |
@pcaversaccio I promise I will make sure snekmate compiles before we merge this. |
We basically want a way to know that we don't need to even call vyper in the first place because none of the files (or their dependencies) have made an update which should change their output. Since subprocessing out to vyper is more expensive than not, having the checksum information in Ape's project metadata helps us avoid unnecessary extra compiler invocations It is also still valuable that vyper does this check to, but would be great if that information were made accessible back to how Ape stores it's own project cache (which may include multiple compilers to check for) |
Let's see if this is even a problem first as well. This is still in development, but getting closer each day. If we can handle Solidity, I think we can handle Vyper. |
0.4.0 made a big shift to absolute file locations from std json input, this is very different than how solidity or vyper (in the past) has handled file inputs but yeah, calling |
since dependencies can change even if source code (of the compilation target) has not changed, there are two surefire ways to do this. one is going to be to call the vyper compiler and get the integrity hash (this only runs the frontend analysis passes, which based on latest benchmarks can process roughly 1000 lines of source code per second). one thing i can do (and i was thinking about doing this anyway) is to move the compiler's import analysis into an even earlier phase, so basically the only work the compiler does to get the integrity hash is resolving imports + hashing. but it would be a bit nontrivial to implement. the other way is to package all the project files into a single package (like standard json input) and check that nothing in the package has changed. |
we essentially do import analysis to build the graph of the project, so this would be really nice to expose. we may in the future execute multiple compiler processes in parallel if we determine (based on the graph) that there are disjoint sets within the project (allowing us to parallelize compiler invocations). an obvious first version of this would be to just parallelize invocations for vyper and solidity files within the same project |
there isn't really work-sharing between different compilation targets. like if X imports A, and Y imports A, each one will re-compile A from scratch. so you can just always parallelize compilation targets. |
how do we know what the targets are? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Doesn't have yo be here, but some tests using modules from snekmate would be great
oh i missed this -- targets are whatever the user asks for. (anything in |
we don't require the user to manually specify their targets, typically we try to identify the most efficient way to compile everything so that when the user goes and asks for a specific target (e.g. requiring the user to specify all of their targets strikes me as poor developer experience. |
So I tested the latest commit and snekmate contracts compile: pcaversaccio/snekmate#244. Two notes:
Collecting vyper~=0.3.7 (from ape-vyper==0.1.0b3.dev69+g6154453)
Downloading vyper-0.3.10-py3-none-any.whl.metadata (6.7 kB) This is unfortunate since I need to uninstall Vyper and reinstall the new Vyper later again (otherwise I have Vyper
|
We should probably only have a lower pin for the flattener CLI, or as low a pin as we can manage Wonder if it makes sense as an extra (and don't add flattener CLI if vyper not installed?) Spitballing here |
This was my thought as well. Another option I was going to look into was check what was being imported from vyper and see if i can copy it in manually (provided it is the same across all versions). It would save a lot of headache and the code won't change much... Either way, can we do this in another PR? |
sounds good, there is a workaround for now (just re-install vyper afterwards)
can you mark this in an issue? |
|
What I did
Make it so can compile on 0.4.0
How I did it
Had to use absolute paths.
How to verify it
Checklist