added armur ai page

[AkhilSharma90]

Project Abstract

Our project Armur A.I is focused on training Open Source Large Language Models with smart contract vulnerability data to be able to detect security issues in smart contracts.
We have a successful working model for Solidity smart contracts and are building one for Move (Aptos).
Building a similar solution where A.I models are trained with Ink! smart contracts for the Polkadot community will ensure that the ecosystem becomes more secure, more trust is generated and gets a higher amount of traction and investment and this is what we all want.

Grant level

• Level 1: Up to $10,000, 2 approvals
• Level 2: Up to $30,000, 3 approvals
• Level 3: Unlimited, 5 approvals (for >$100k: Web3 Foundation Council approval)

Application Checklist

• The application template has been copied and aptly renamed (project_name.md).
• I have read the application guidelines.
• Payment details have been provided (bank details via email or BTC, Ethereum (USDC/DAI) or Polkadot/Kusama (USDT) address in the application).
• The software delivered for this grant will be released under an open-source license specified in the application.
• The initial PR contains only one commit (squash and force-push if needed).
• The grant will only be announced once the first milestone has been accepted (see the announcement guidelines).
• I prefer the discussion of this application to take place in a private Element/Matrix channel. My username is: @_______:matrix.org (change the homeserver if you use a different one)

[CLAassistant]

All committers have signed the CLA.

[Noc2]

Thanks a lot for the application, and sorry for the late reply here. Let me share this application also with @bhargavbh, who previously looked into ink! smart contract vulnerabilities. I have one question here: You say all your A.I. models are open source, but is any of your work so far actually open source? I couldn't find a GitHub organization. My main concern here is that the dataset that you will create as part of the first milestone is only useful for you and can not easily be reused by others. Another big concern that I have is that your project is simply too early for our ecosystem, given that there will be very little data to train your models.

[bhargavbh]

Thanks for the application. I concur with David's concerns. What were the approx size of dataset and manual annotations (avg for each vulnerability class) needed for other smart-contract languages to create a robust (non overfitting) model? I feel there are many low hanging fruits for deterministic semantic analysis before diving in to ML based approaches.

[semuelle]

Thanks for the application.

When you say that you have "built a successful A.I powered auditing tool for solidity files", can you elaborate how you define success? And can you share the training and test sets and some raw evaluation numbers (beyond "80% accurate")?

[AkhilSharma90]

You're right, the A.I models are open source but our work up until now isn't. This is because all the work we've done until now has been for solidity smart contracts and since we haven't raised a Pre-Seed round yet, we haven't open sourced the work as this is the I.P edge we want to demonstrate to investors.
But when we work with Polkadot, all the work will be open-sourced since in a way Polkadot will be our investors for this work and Polkadot is funding it. We will sign all your agreements that state we have to share our work with the entire community.

[AkhilSharma90]

Thanks for the application. I concur with David's concerns. What were the approx size of dataset and manual annotations (avg for each vulnerability class) needed for other smart-contract languages to create a robust (non overfitting) model? I feel there are many low hanging fruits for deterministic semantic analysis before diving in to ML based approaches.

Here's the process we followed for Solidity smart contracts -
(our process is very much human feedback based and we label the data carefully, there's no unsupervised learning here)

1. We first trained the model with already classified issues that were available in the SWC registry. Meaning, this registry already had 40 vulnerabilities, properly classified with one code sample having the issue and another code sample that had the issue fixed. this is where we started as it was the low hanging fruit. Now please note that already the A.I model we used already understood programming languages so it already has a lot of context and you benefit from few shot learning. After just this first step which is just 40 examples - model was able to detect issues reasonably.
2. we used tools like slither and manticore, passed multiple smart contracts through them and trained the A.I models with the results. We did this for about 700 smart contracts.
3. We then used Chat-GPT, LLaMa and Bard, presented these with smart contract code samples and asked them to find issues. The output from these models was used as the "Expected output" and the input code sample was used as the "input" to train our models.
4. we then found 2,000 audit reports from github from various companies, converted PDF to JSON and trained the A.I models with the code of the smart contracts as the input and the "insights" sections of the audit reports as the expected output.

After these 4 steps, the A.I models work really well to create a non-overfitting model.
since we used deterministic tools in step 3, the A.I model can be thought of having the skills to detect issues that a semantic anaylser would detect but would also have the capability to go beyond.

The main benefit here over a hard-coded semantic analysis tool is that we won't EVER have to write "extensions" or "plugins" for the existing code base. That's the benefit you get with an A.I driven approach - we simply need to train it in the future, rather than "maintaining" a code-case that needs to be extended.
Not sure if you're following the issues that the Slither community is facing - everytime there's a new issue, new extensions end up conflicting with the old ones, this is why there are barely any updates even when months go by.
Whereas we can train A.I models every day with new vulnerabilities :D

[AkhilSharma90]

Thanks for the application.

When you say that you have "built a successful A.I powered auditing tool for solidity files", can you elaborate how you define success? And can you share the training and test sets and some raw evaluation numbers (beyond "80% accurate")?

Hi Semuelle,
regarding accuracy -
As you know, we have been working on this problem since the past 9 months and have gone through the Techstars accelerator and are currently going through the Outlier Ventures accelerator.
By now we have worked (audited code for) about 60 companies - these are early stage web3 startups.
We strongly believe that accuracy metrics are only relevant if the customer feels that. In our case, once we created an audit report for these 60 companies, we sat down with the engineering team to understand what we got right and we didn't.
Here are the learnings -

1. Clients feel that our tool is on an average ~75% accurate in finding vulnerabilities that have been properly classified and are known.
2. We are by no means trying to replace manual auditors, in fact, we save them time. Also, for early stage web3 startups, we save a lot of time and money since now they don't have to go to a manual auditing firm for every small thing and atleast the basic level of issues can be found and corrected.
3. The clients feel empowered since now they can do wwayyy more audits. This is essentially, a shift-left of auditing or democratization of auditing.
4. Clients have reported hallucinations and false-positives and that's what makes up the other 25%. The only way to fix this is to work with more data, but also tighter human feedback loop, which we're currently prioritizing.

[AkhilSharma90]

Thanks a lot for the application, and sorry for the late reply here. Let me share this application also with @bhargavbh, who previously looked into ink! smart contract vulnerabilities. I have one question here: You say all your A.I. models are open source, but is any of your work so far actually open source? I couldn't find a GitHub organization. My main concern here is that the dataset that you will create as part of the first milestone is only useful for you and can not easily be reused by others. Another big concern that I have is that your project is simply too early for our ecosystem, given that there will be very little data to train your models.

Have answered all the questions above. I just wanted to highlight something -

We have been referred by Gavin Wood, kindly check with him once as well. This is the reason we applied for the grant. Our original plan was to become a para-chain but Robin Ejsmon-Frey and Nico Morgan suggested that grant could be the right route.
In addition, I've been in touch with Bill Laboon and discussed this with him. In addition, I've been in touch with Tim Dobie and Santiago Balaguer.
So we're in touch with people across the entire Polkadot eco-system - Parity, Square One, Substrate and Web3 Foundation etc.
Need your support so that we're able to build with you guys :D

[AkhilSharma90]

please let me know if I should make changes in the original submitted proposal or if these comments are enough?

[semuelle]

Thanks for the updates. Could you update the specification for the default deliverables? For example, there is no code delivered in M1, so I don't see how inline docs and Docker are relevant.

[semuelle]

Looking at the other milestones and deliverables, I have to say they don't really fit the grants program.

• We don't pay for deployment, unless it's an infrastructure project. And since you are planning to make the models proprietary, I don't see how that would work.
• What do you mean by testing and deploying the model(s) if you only train them in the next step? I assume this is an API you are talking about? Please review all deliverable specifications, as they should be something verifiable and reusable. For example, if you want "model selection" to be a deliverable, the test procedure and results should also be a part of the deliverable.
• 16 person months for training and testing of models seems excessive.

[AkhilSharma90]

Looking at the other milestones and deliverables, I have to say they don't really fit the grants program.

• We don't pay for deployment, unless it's an infrastructure project. And since you are planning to make the models proprietary, I don't see how that would work.
• What do you mean by testing and deploying the model(s) if you only train them in the next step? I assume this is an API you are talking about? Please review all deliverable specifications, as they should be something verifiable and reusable. For example, if you want "model selection" to be a deliverable, the test procedure and results should also be a part of the deliverable.
• 16 person months for training and testing of models seems excessive.

Hey Semuelle,

I think there's a lot of confusion.
You're quoting things exactly opposite of what I have mentioned on the comments above.

1. I have clearly mentioned that there are going to be NO proprietary models. The models we currently use are Open Source and all of our work for Polkadot will be 100% open source and will be for the community. Just to clarify further - the work we've done on solidity smart contracts is NOT open source (since we funded it oursevles) but the work done for Polkadot will be - since we will have the grant to fund this.

2. We never asked you to pay for deployment and I obviously know how Polkadot works and that you don't pay for deployment, no idea why you thought this is being asked.

3. By testing and deployment we mean - we use some sample datasets and check which ones of the open source models are the best suited for ink! smart contracts. we've noticed that GPT-NEO-X works really well for Viper and Falcon-40B works really well for Solidity. Now there are new A.I models each day in the market (for example LLaMa 2 just came out). Based on the latest ones in the market at the time the grant get approved, we will test the ones with sample datasets for ink! smart contracts and find the best ones for the use case.

4. Please understand, we know what we're doing really well. We've already made it through the Techstars accelerator, Outlier ventures accelerator, we are working closely with Aptos now and soon will be raising our pre-seed.

5. Regarding your comment on 16 person months being excessive - how many companies do you know who've successfully done this? Please let me know.
   The reason we're asking for this time is because we're quite pragmatic and have delivered enterprise projects before so this timelines includes buffer periods, includes time for experimentation and testing. Also, creating the datasets is a difficult task as it requires a lot of research and that's where the max time will go.

Thanks,

[Noc2]

But when we work with Polkadot, all the work will be open-sourced since in a way Polkadot will be our investors for this work and Polkadot is funding it. We will sign all your agreements that state we have to share our work with the entire community.

That's good to hear. Could you try to make this more clear as part of the milestone tables? The size of the current ink smart contract dataset is also still a concern for me. So the application might be a little bit too early.

[semuelle]

Hi @AkhilSharma90,

ad 1: I think we are using the term 'model' differently. To me, an open source model is a trained model whose parameters and algorithms are open source. For example, to open source my neural network, I would publish its weights and biases. When you say you have "trained open source models", I hear "we have published the models we have trained", but - correct me if I'm wrong - you are referring to the fact that the software you have used to train the models is open source. Your website refers to your "proprietary technology", so I assumed you were not planning to publish the trained model. Perhaps you can clarify this.

ad 2: deployment is the title of your second milestone and listed as one of your deliverables.

ad 3: I think paying for this kind of deployment would be fine if you published the results of the work, the models and ideally the test results. 'Deployment' is usually used for backends and smart contracts around here, which we don't fund.

ad 5: It's clear to me and everyone reading your proposal that you have already put a lot of work into your project and there are not many people who could and would pursue it. However, given the novelty of the concept, I would prefer to see some hard data on its accuracy, precision and market interest, so a more iterative approach would make more sense to me. Also, since you mention datasets: there is a milestone for creating datasets and it's listed with three person-months.

Lastly: by "we need help of the Polkadot community", do you mean this grant (application) or are you planning to involve the community in different ways?

[keeganquigley]

Thanks for the application @AkhilSharma90 in addition to above comments:

• Can you update to specify which license you will use? Currently, it still lists all four.
• The deliverables still contain mainly boilerplate text. Is Docker really necessary for all milestones?
• What kind of tests will be created and how can we evaluate them?
• For milestone 3, how can we effectively evaluate the human feedback loop that is mentioned?

[semuelle]

pinging @AkhilSharma90

[AkhilSharma90]

Hi team, Apologies for the late reply, we have our friendly investor pitch day at Outlier Ventures on 11th, I got busy fixing the deck and the data room :) I'll reply to all the comments and make changes as well

…

On Wed, Aug 9, 2023 at 7:49 PM Sebastian Müller ***@***.***> wrote: pinging @AkhilSharma90 <https://github.com/AkhilSharma90> — Reply to this email directly, view it on GitHub <#1865 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGKJCNPWKUOBMT6CAU2V7WLXUOL7TANCNFSM6AAAAAA2YEJMHM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[AkhilSharma90]

Thanks for the updates. Could you update the specification for the default deliverables? For example, there is no code delivered in M1, so I don't see how inline docs and Docker are relevant.

yes, i will do this and push the changes

[AkhilSharma90]

But when we work with Polkadot, all the work will be open-sourced since in a way Polkadot will be our investors for this work and Polkadot is funding it. We will sign all your agreements that state we have to share our work with the entire community.

That's good to hear. Could you try to make this more clear as part of the milestone tables? The size of the current ink smart contract dataset is also still a concern for me. So the application might be a little bit too early.

got it, we need atleast 5-6 code examples per categorized vulnerability to be successful. We can additionally build this dataset with you if we're too early. We're passionate about the problem statement. Being an early mover will help us a lot. If the timelines can be relaxed for us to let us help you build the datasets, will be an awesome collaboration

[AkhilSharma90]

Hi @AkhilSharma90,

ad 1: I think we are using the term 'model' differently. To me, an open source model is a trained model whose parameters and algorithms are open source. For example, to open source my neural network, I would publish its weights and biases. When you say you have "trained open source models", I hear "we have published the models we have trained", but - correct me if I'm wrong - you are referring to the fact that the software you have used to train the models is open source. Your website refers to your "proprietary technology", so I assumed you were not planning to publish the trained model. Perhaps you can clarify this.

ad 2: deployment is the title of your second milestone and listed as one of your deliverables.

ad 3: I think paying for this kind of deployment would be fine if you published the results of the work, the models and ideally the test results. 'Deployment' is usually used for backends and smart contracts around here, which we don't fund.

ad 5: It's clear to me and everyone reading your proposal that you have already put a lot of work into your project and there are not many people who could and would pursue it. However, given the novelty of the concept, I would prefer to see some hard data on its accuracy, precision and market interest, so a more iterative approach would make more sense to me. Also, since you mention datasets: there is a milestone for creating datasets and it's listed with three person-months.

Lastly: by "we need help of the Polkadot community", do you mean this grant (application) or are you planning to involve the community in different ways?

1. Since for vulnerabilities, the datasets are not in millions/ billions, what this means is we have to fine-tune A.I models that are pre-trained. I apologize, I haven't made this clear, this approach is becoming quite common now, especially after this very famous research paper -> https://arxiv.org/pdf/2306.12338.pdf. Our methodology is based heavily on this.

2. We will deploy to something like AWS for the community to use. The work and A.I models will be open sourced by us and any one across the world can deploy them on their own system and run these. We have to find the right model, deploy it using sagemaker and train it with the datasets we create with your help or that are existing at the moment. We will keep the A.I model deployed and the community can use ours through a light front end client (that we can also develop) or they can deploy it on their own. We will also open source the entire method of deploying it with sagemaker and share all steps

3. All reports will be shared with you, all data and datasets will be shared with you, all work done by us for this grant, is owned by the Polkadot community or the people, it's all open source. With the grant you're paying for the part time salaries of data scientists, A.I engineers and MLOps resources that we already worked with to train our model with solidity data.

4. When we talk about "datasets" and timeline with it, we mean that we are assuming that the list of vulnerabilities already exist and there are plenty of code samples for each vulnerability (atleast 4-5). And using this, we simply create a "dataset" which has 3 fields (required for Alpaca models) -> input (which is going be our code), prompt (which is something like - give me a list of vulnerabilities in this code) and expected output (which is the vulnerabilities in that code).

Your team member, NOC2 mentioned earlier that the dataset on vulnerability is too small, the term "dataset" that he's using is actually meant to refer to the raw data of the vulnerabilities and code samples that might exist for us to be able to create a dataset from.
I don't have visibility on this and since you're more deeply involved in the community and ecosystem, you would have a better idea if we're too early.
And my reply to this is that, if you feel that even this data is less, we would love to help you build this as well. Even though we are very early for this and since we know that it will take a while for us to build this data, we would like to be involved at an early stage with you, because we all know this will be useful in the near future.
Now this might mean relaxing the 3 month timeline for creating the dataset or any other way. Our priority is to get to work with you on this problem, which is an extremely important problem and will have quite big effects - democratizing audits.

6. When I mention help of the polkadot community, I mean their help in building the vulnerabilities data, if this could be a cause that we could lead and more developers from the community also contribute to this. That's the vision.

[AkhilSharma90]

Thanks for the application @AkhilSharma90 in addition to above comments:

• Can you update to specify which license you will use? Currently, it still lists all four.
• The deliverables still contain mainly boilerplate text. Is Docker really necessary for all milestones?
• What kind of tests will be created and how can we evaluate them?
• For milestone 3, how can we effectively evaluate the human feedback loop that is mentioned?

Ok I will make all these changes to the application.
Could you also go through my above comments and let me know if you have more questions

[AkhilSharma90]

Thanks for the application @AkhilSharma90 in addition to above comments:

• Can you update to specify which license you will use? Currently, it still lists all four.
• The deliverables still contain mainly boilerplate text. Is Docker really necessary for all milestones?
• What kind of tests will be created and how can we evaluate them?
• For milestone 3, how can we effectively evaluate the human feedback loop that is mentioned?

Hi Keegan,
I have updated all the milestones to remove boilerplate text and added a lot more details in each of the milestones.
regarding questions 3 and 4, I have added details in the milestones itself, please let me know if it's still not clear and I'll reply in comments as well.

[keeganquigley]

Thanks for the changes @AkhilSharma90 I appreciate the added technical details. However, it looks like the mandatory deliverables (0a. - 0e.) were removed from all milestones. These should stay, apologies if I wasn't clear on that. The 0d. Docker section can be removed if it's not applicable, and the 0e. blog article is only required for the last milestone. IMO would be great to have that after the grant is completed.

0b. and 0c. should remain, for "Documentation" and "Testing & testing guide" respectively. That way we have a tangible way to evaluate the work that has been accomplished. Not sure what these tests would consist of for this kind of project though. For example I'm still not exactly sure how we would evaluate milestones 3 & 4. Will there be some kind of guide or tutorial that walks us through how to interact with the A.I. models?

[AkhilSharma90]

Thanks for the changes @AkhilSharma90 I appreciate the added technical details. However, it looks like the mandatory deliverables (0a. - 0e.) were removed from all milestones. These should stay, apologies if I wasn't clear on that. The 0d. Docker section can be removed if it's not applicable, and the 0e. blog article is only required for the last milestone. IMO would be great to have that after the grant is completed.

0b. and 0c. should remain, for "Documentation" and "Testing & testing guide" respectively. That way we have a tangible way to evaluate the work that has been accomplished. Not sure what these tests would consist of for this kind of project though. For example I'm still not exactly sure how we would evaluate milestones 3 & 4. Will there be some kind of guide or tutorial that walks us through how to interact with the A.I. models?

Hi Keegan,

Please check now, requested changes done

Thanks,

[AkhilSharma90]

@keeganquigley all ok? please let me know if any more details required

[semuelle]

We then used Chat-GPT, LLaMa and Bard, presented these with smart contract code samples and asked them to find issues. The output from these models was used as the "Expected output" and the input code sample was used as the "input" to train our models.

This sounds like you are using the untrained model output as the expected output for your training, effectively training on incorrect data. Can you elaborate?

Do you have a list of vulnerabilities your Solidity model recognises? I just had a look at the article you linked above, and it lists attacks like "Fake tokens"/"Fake contracts" and "Governance attack"? Those are largely unrelated to the source code of the contracts, so I'm curious how those would be discoverable.

[semuelle]

"whimsical"?

[AkhilSharma90]

We then used Chat-GPT, LLaMa and Bard, presented these with smart contract code samples and asked them to find issues. The output from these models was used as the "Expected output" and the input code sample was used as the "input" to train our models.

This sounds like you are using the untrained model output as the expected output for your training, effectively training on incorrect data. Can you elaborate?

Do you have a list of vulnerabilities your Solidity model recognises? I just had a look at the article you linked above, and it lists attacks like "Fake tokens"/"Fake contracts" and "Governance attack"? Those are largely unrelated to the source code of the contracts, so I'm curious how those would be discoverable.

Hi Semuelle,

Please note that while we have a working product for Solidity (and have gone through multiple stages of technical due dilligence from the two accelerators we are a part of),
we applied to the Polkadot grant to experiment and build similar solutions for the eco-system. The keyword being "experiment" since we assumed that the grants are to fund "experiments" that may lead to success and solutions that contribute exponentially to the eco system.

However, in the past few weeks while interacting with you and the Polkadot team, we feel that this is taking a very different direction. The questions that are being asked to us are as if we have a working product for Polkadot (and we have been answering the questions too).

The thing is, we will have to experiment quite a bit and are not sure if the same methodology will work here.

We were considering this as a collaboration to build something awesome with an experimentative approach, but you're going a lot into unnecessary details and nitty gritties.

What's happening here is the same thing that's happened in all my previous interactions with the Polkadot community - long stretched out conversations that are always inconclusive. Also, there's always analysis paralysis and decision fatigue.

We (Armur) are a fast growing and fast moving startup and have multiple priorities, we thought collaborating with Polkadot could lead to great results and we also believe that all the details you require to take a decision are already with you by now.
There is no end to clarifications, questions that can be asked to stretch the conversation.

It seems like what we want to build is clearly not a priority for you at the moment and if you feel this doesn't help the Polkadot ecosystem much, we're happy to close the application.

Now regarding the questions you've asked -

1. You're again assuming something that's wrong. Chat-GPT, bard etc. can already audit smart contracts (atleast that's the case with solidity), training on their outputs ensures you have your base covered.
2. We're able to detect all vulnerabilities from the SWC registry - https://swcregistry.io/
3. Regarding the article - you're again getting it all wrong. I mentioned that our approach is heavily based on the research paper, I never said that we're following the same approach. I have already explained our approach to you in the milestones.

We can do this endlessly and this is wasting our time and yours, if you feel this isn't a priority, please just let us know instead of making incorrect assumptions and asking irrelevant questions.

Thanks,

[keeganquigley]

Thanks @AkhilSharma90 for your answers. I will try to address some of your concerns below:

• It is normal for committee members to leave comments/questions during the application review. These questions are meant to open up a discussion. One reviewer asking for a specific change is not required to continue the review; it is up to the applicant to decide which suggestions to adopt from individual reviewers.
• We do indeed expect teams to have a solid vision of what they would like to build, with the implementation details already planned out. I'm not sure how experimental this makes the grants program.
• The grants committee members all come from different backgrounds and areas of expertise, therefore we ask questions in an effort to better understand the project and get to know the team better. We may not always ask the right questions. In my opinion, making disparaging remarks about the process and the reviewers for the questions they are asking may not be the best approach in getting the application approved.
• For teams who we haven't worked with before, we usually ask for a smaller scope at first, before going for a level 3. Considering the unknown variables that you mentioned, I think it might be best to break down the grant into a smaller PoC, and then perhaps a follow-up grant later, once more of the details have been cemented. Therefore I will pass on approving the application for now.

[semuelle]

The thing is, we will have to experiment quite a bit and are not sure if the same methodology will work here.

In that case, I recommend a smaller, initial grant to make some tests and write some reports, then to apply for a follow-up grant if the results are promising. You could break it down into a POC as Keegan suggested, or turn it into a research grant and deliver a report. I do think a dataset of ink! vulnerabilities and the results of training a model on them would be interesting.

if you feel this isn't a priority, please just let us know instead of making incorrect assumptions and asking irrelevant questions.

I asked whether output of a model is the input for your training, which I took from how you described your pipeline. Which would be unusual, as your ground truth would be highly flawed. These are very simple and relevant questions to ask from a machine learning project. Perhaps you can illustrate your training and test pipeline in more detail in the application to avoid further questions.

You're again assuming something that's wrong.

I said "it sounds like", and asked you to elaborate. Very few assumptions here.

Chat-GPT, bard etc. can already audit smart contracts, training on their outputs ensures you have your base covered.

Going by table 4 of the article you linked¹, Chat-GPT has a precision of 78.7% and recall of 69.8% for a binary classification task. The false positives for the non-binary task (which you are aiming for) aren't reported, but we can assume precision and recall are worse. Training on their output means your ground truth is flawed. I think these metrics are highly relevant, so that we don't give users a false sense of security.

We were considering this as a collaboration to build something awesome with an experimentative approach, but you're going a lot into unnecessary details and nitty gritties.
What's happening here is the same thing that's happened in all my previous interactions with the Polkadot community - long stretched out conversations that are always inconclusive. Also, there's always analysis paralysis and decision fatigue.

I'm sorry you are having a hard time with the community. I cannot speak for others, but it is our responsibility to make sure the funds go to projects that further the ecosystem. This is a technical program, so making sure that the concept is technically sound is a big part of that process.

Footnotes

1. David, I., Zhou, L., Qin, K., Song, D., Cavallaro, L., & Gervais, A. (2023). Do you still need a manual smart contract audit?. arXiv preprint arXiv:2306.12338. ↩

[AkhilSharma90]

@keeganquigley

• Those were not disparaging remarks, just facts about my previous interactions. Because all the details have already been shared and there's just a lot of delays.
• I assumed that grants were for experimentation. There is a clear path that's already been shared with you. I admit that the results could be unpredictable but atleast we (Polkadot community) would be ahead of our time for something that's eventually going to happen.
• I completely understand if you want to pass on approving the application. Please note that the Polkadot community would have really benefitted from this project, as this is the future. My request would be to not have a narrow, myopic view that ends up hurting the community. We truly believe Polkadot needs this badly and we're willing to put in the work and we're highly execution driven (we have a proven track record).
• Regarding POC, we already have a POC for solidity. you can check on our website, www.armur.ai and click "launch app", test it out if you wish. You would have got something similar for Polkadot but everything open source.
• If you wish to test it our, follow these instructions - https://www.loom.com/share/d433d443d1584bc3ab00c454646dd0b6?sid=4e0f24c1-405b-4274-b3ff-c0343da8f934

[AkhilSharma90]

@keeganquigley @semuelle
Guys, we're very serious about working on this and contributing to the community.
Money isn't the priority for us.
You can delay our payments, like pay us after 6 months if you want. We want to work with you guys / collaborate on this problem. That's what's more important.

1. creating the dataset for ink! vulnerabilities
2. finding the right models
3. finetuning them
   I know that you guys need it in the near future.

[takahser]

@AkhilSharma90 Thanks for the proposal, but I have some of the same reservations as other committee members who shared their feedback.

The ink! dataset seems too sparse right now to train effective models. I'd also want more technical details on your existing proposal. The budget and timeline also seem high, especially given the unknowns.

Also, open-sourcing your Solidity outputs could also help ease IP concerns.

Since none of the committee members who gave feedback so far seem convinced either, I'm going to close it for now. But you're always welcome to apply for a new grant for another idea or a revised proposal, that incorporates all the critiques that were raised.