Add delightfuldot application

[sinzii]

Project Abstract

delightfuldot is a lightweight alternative solution for @polkadot/api that aims to address its high memory consumption issue in helping dapps to connect to a large number of Substrate-based networks effectively and efficiently.

Grant level

• Level 1: Up to $10,000, 2 approvals
• Level 2: Up to $30,000, 3 approvals
• Level 3: Unlimited, 5 approvals (for >$100k: Web3 Foundation Council approval)

Application Checklist

• The application template has been copied and aptly renamed (project_name.md).
• I have read the application guidelines.
• Payment details have been provided (bank details via email or BTC, Ethereum (USDC/DAI) or Polkadot/Kusama (USDT) address in the application).
• The software delivered for this grant will be released under an open-source license specified in the application.
• The initial PR contains only one commit (squash and force-push if needed).
• The grant will only be announced once the first milestone has been accepted (see the announcement guidelines).
• I prefer the discussion of this application to take place in a private Element/Matrix channel. My username is: @____:matrix.org (change the homeserver if you use a different one)

[CLAassistant]

All committers have signed the CLA.

[keeganquigley]

Thanks for the application @sinzii very interesting. I think it would be nice to have an alternative to the Polkadot.js API.

• Can you check out this other wrapper project sub-API by KodaDot and briefly compare to it?
• I'm not seeing that a lot of people have adopted the Coong Wallet so far; do you plan on continuing to maintain that as well?
• Considering the above how do you plan to market the API to get devs to switch to it?

Thanks!

[sinzii]

@keeganquigley thank you for taking the time to review the proposal and for the questions

Can you check out this other wrapper project sub-API by KodaDot and briefly compare to it?

As far as from what I've taken a look, sub-API is a helper to help manage connections to multiple Substrate-based networks, this is done via a mapping between a network endpoint/prefix and a @polkadot/api ApiPromise instance. This is helpful when you need to manage a lot of ApiPromise instances since you can easily set them up and quickly/lazily get an instance by providing the network endpoint or prefix.

Most of the dapps that need to connect to multiple networks would also need this API management, for example, SubWallet is also using a similar approach to manage their connections to networks.

But under the hood, they are still using @polkadot/api as a mean of connection so they are still being impacted by the aforementioned high memory consumption & slow startup time issues.

delightfuldot is a replacement for @polkadot/api, where we offer similar functionalities but without those issues and dapps can choose to connect to networks via @polkadot/api or delightfuldot. We could probably redefine again the mapping as below:

type ApiType = ApiPromise | DelightfulApi;
const connectionMap: Map<string, ApiType> = new Map<string, ApiType>()

I'm not seeing that a lot of people have adopted the Coong Wallet so far; do you plan on continuing to maintain that as well?

This is understandable and to what we expected since we haven't made any marketing effort for Coong Wallet. Coong Wallet is now still in its early phase of development/incubation, we plan to add more & more useful features for Coong Wallet before we introduce it to the mass, and delightfuldot is a stepping stone for those.

As a website-based wallet, users tend to open multiple tabs of Coong Wallet and we don't want to see each tab consume hundreds of MB or even GB of RAM on users' devices because of the high-memory consumption issue hence the need for a better & lightweight tool such as delightfuldot for connecting a large number of networks efficiently. We also believe a lot of other dapps in the ecosystem are also in our shoes so we decide to go ahead with this proposal so potentially others could benefit from it when the whole idea becomes a reality.

Considering the above how do you plan to market the API to get devs to switch to it?

We haven't put a lot of thought into this, but I think we can start with some of the below ideas:

• Prepare a good/clear documentation/website.
• Reach out to dapps/wallet in the ecosystem that has the issue with @polkadot/api when connecting to a large number of networks to introduce it (potentially SubWallet, Polkadot JS Plus Extension, Kodadot ...)
• Start sharing the ideas/work on Polkadot Forum, Twitter, Telegram, or Matrix channels in the ecosystem so people could be aware and start trying around.
• ...

With a vision to become an essential part of the ecosystem, delightfuldot will also be a long-term project for us aside from Coong Wallet, we're also having other ideas/plans coming along with it, so we'll update more in the future.

[keeganquigley]

Thanks @sinzii I had just found it in passing so thanks for looking into it. I appreciate the thorough answers. I will go ahead and mark the application as ready for review and ping the rest of the committee members. In the meantime, it looks like the image links have broken. Can you fix these? Thanks!

Also, have you talked to any wallet developers or other network intensive dapp developers to see if this is a common concern and that this solution is needed?

[sinzii]

it looks like the image links have broken.

Ah, thank you @keeganquigley for the catch, I've just fixed that!

have you talked to any wallet developers or other network intensive dapp developers to see if this is a common concern and that this solution is needed?

Last year, we had a chance to collab with SubWallet team to review the wallet to improve its performance and stability. At that time, the memory consumption of the wallet was quite high (I don't remember the exact number), and the team had made some effort to limit the usage of @polkadot/api and introduce the toggles to turn on/off the connection to networks (so the wallet doesn't need to connect to all of the networks at the same time, but only to those networks that users have assets on) to somehow mitigate this issue.

We haven't had any chance to talk to other wallets or dapps to have their insights but in general, we believe this is an issue that other dapps/wallets would also running into when connecting to a large number of networks via @polkadot/api

[sinzii]

Let me know if you have any feedback/thoughts on the application, thanks!

[semuelle]

Sorry for the delay, @sinzii. I will review the application and provide some feedback tomorrow.

[Noc2]

Thanks a lot for the application, and I fully agree that we need an alternative to the @polkadot/api. Parity will continue to work on Capi. Nevertheless, I think it makes sense that we also fund external efforts, but it would be nice if this would be somehow coordinated between different wallet providers. This way, you could also potentially apply for larger amounts via the treasury later. Have you already reached out/talked to any other wallet/UI provider?

[sinzii]

Hi @Noc2, I totally understand the concern about the coordination, and to some extent absolutely agree with that.

But we would like to start working on this ourselves at least for the first development phase of the project, as the more people involved in a project, the longer it would probably take in making decisions and moving forward as a whole. We expect to make a lot of decisions about the architecture, API, project setup, and structure, especially for an open-source project there would need a lot of considerations in designing the API so that it would be extensible and open for people to contribute later on. We believe we can move faster starting this ourselves.

At some point in the future, we believe there would definitely need to have coordination between wallet/ui providers/projects as there are people/projects starting to use it and run into problems that need to address/standardize so that it would be easier for them to adopt with their use-cases/needs. We do have plans to talk to other wallets in the ecosystem that are currently relying on @polkadot/api (e.g: SubWallet, PolkadotJS-Plus, ...) to start trying this around and gather feedback to further improve the project when it's in a good shape.

As the maintainer of Coong Wallet, we also understand some of the problems that a wallet/project will likely get into when connecting to a large number of networks, to design the library so that it would be easy for others to use and adapt.

Let me know how that sounds, thanks!

[semuelle]

Thanks for the application, @sinzii. I think it's quite interesting. I am somewhat in the same boat as @Noc2 and @keeganquigley, in that I think this project would require a fairly large user/developer base to be maintained. So getting as many people involved early as possible might be beneficial in the long term. I don't have an outright suggestion, but the idea of coordinating this between wallet providers seems the right way to go.

[sinzii]

Thank you everyone for the feedback and suggestions and in response to that, we're currently in discussions with SubWallet team for potential collaboration/coordination on this project. We'll update more on the plan this week.

[sinzii]

My apologies for the delayed update here, our discussion with SubWallet team has gone a bit more involved with more plans that are out of the scope of this grant. I'll update as soon as we reach an agreement and finalize the whole plan. Thanks!

[keeganquigley]

Sounds good @sinzii thanks for the update, feel free to ping us all once we should take another look.

[keeganquigley]

Hi @sinzii any updates so far? Thanks.

[sinzii]

@keeganquigley Sorry for the delayed reply here, after a prolonged discussion over the past few weeks with several potential approaches, we couldn't really find a good common ground for both teams to move forward in a collaboration unfortunately.

While working in collaboration is a good approach, but for this first iteration, I don't think it works for us for now. So please do let me know if you have any other suggestions to help move this forward, I'm open to reducing the scope of the work to a smaller level to build this up.

With a plan to support 1000 parachains in the future, this further strengthens our belief in a light and fast JS client solution for Polkadot ecosystem.

Thank you!

[keeganquigley]

Thanks @sinzii I appreciate the update. That's a shame, sorry to hear the collaboration didn't work out. I would personally be willing to go forward with a smaller scope for perhaps a level 2 PoC. It's up to you whether you'd want to combine the core functionalities into one milestone or keep it split into two; feel free to refactor the application.

For now I will go ahead and remove the on hold status. Happy to schedule a call as well, if you'd like to discuss further. If so you can book a slot here.

[github-actions]

CLA Assistant Lite bot All contributors have signed the CLA ✍️ ✅

[sinzii]

I have read and hereby sign the Contributor License Agreement.

[sinzii]

Hi everyone, I've just reduced the proposal scope to a level 2 grant, the work will mostly be focused on building foundational modules and core functionalities. Please take your time to review the application again and let me know if you have any feedback. Thanks!

[keeganquigley]

Thanks for making the changes @sinzii this looks good to me, happy to approve it.

[sinzii]

@keeganquigley Thanks a lot for your support & approval!

[takahser]

@sinzii in general this looks like an interesting alternative to the existing tooling.
I just have 2 remarks:

1. In M1 you mention "APIs to inspect pallets' events & errors". Does this mean you can only query past events or is it also possible to listen to them in real-time?
2. Another thing that is unclear to me is whether there would be any key management functionality, i.e. generating new keys, adding existing keys, signing with keys, etc. Could you clarify on that as well?

[sinzii]

@takahser Thank you for taking the time to review the proposal and for the questions:

In M1 you mention "APIs to inspect pallets' events & errors". Does this mean you can only query past events or is it also possible to listen to them in real-time?

By APIs to inspect pallets' events & errors, I meant to get/inspect events and errors for each pallet from the metadata. These are similar to the @polkadot/api APIs to inspect events & errors.

The ability to query past/current events or listen to real-time events will be supported via storage query to system pallet which will be supported in the APIs to execute pallets' storage queries part. For example, to query/subscribe to events for current block, we'll call the API: api.query.system.events, more details about this can be found here

Another thing that is unclear to me is whether there would be any key management functionality i.e. generating new keys, adding existing keys, signing with keys, etc

In the scope of this grant, we will not provide these functionalities, dapp developers can use existing packages such as @polkadot/keyring, @polkadot/ui-keyring for those purposes. We'll mostly focus on the APIs to interact with a Substrate blockchain.

On that note, we also do notice some issues with existing packages for generating & managing keys, so we do have plans to implement our own solution for those functionalities in the future.

[sinzii]

@keeganquigley @Noc2 I've just pushed a commit to fix a small typo, so you may need to reapprove the PR. My apologies for the inconvenience.

[takahser]

@sinzii thanks for the prompt reply.

The ability to query past/current events or listen to real-time events will be supported via storage query to system pallet which will be supported in the APIs to execute pallets' storage queries part.

That's great. Could you make this clear in the proposal?

dapp developers can use existing packages such as @polkadot/keyring, @polkadot/ui-keyring for those purposes.

That's fine by me, if these are downstream dependencies for your package. Can you make it clear in the proposal that this will be the case? Just to make sure the integration will be seamless with your package.

After updating on these 2 issues I'll be happy to add my approval as well.

[sinzii]

@takahser I've just updated the proposal, let me know if that addresses your concerns, thanks!

[takahser]

@sinzii thanks for the clarification, happy to approve as well.

[nikw3f]

I would very much like to support polkadot js alternative. One of my wishlist items are not downloading whole metadata for parsing simple rpc calls, see if its possible. I had discussion reagarding this with Rob. He said its possible with merkel proof. Happy to approve.

[semuelle]

Standing in for David and Keegan.

[github-actions]

Congratulations and welcome to the Web3 Foundation Grants Program! Please refer to our Milestone Delivery repository for instructions on how to submit milestones and invoices, our FAQ for frequently asked questions and the support section of our README for more ways to find answers to your questions.

Before you start, take a moment to read through our announcement guidelines for all communications related to the grant or make them known to the right person in your organisation. In particular, please don't announce the grant publicly before at least the first milestone of your project has been approved. At that point or shortly before, you can get in touch with us at [email protected] and we'll be happy to collaborate on an announcement about the work you’re doing.

Lastly, please remember to let us know in case you run into any delays or deviate from the deliverables in your application. You can either leave a comment here or directly request to amend your application via PR. We wish you luck with your project! 🚀

[sinzii]

Awesome, thank you everyone for your support! 🚀

One of my wishlist items are not downloading whole metadata for parsing simple rpc calls, see if its possible. I had discussion reagarding this with Rob. He said its possible with merkel proof.

@nikw3f I'm very curious about the merkel proof that Rob mentioned, would you mind sharing it with me if he did elaborate on that term and approach to achieve that? Metadata is one thing we need to optimize to achieve a better performance, so aside from the proposed approach, we're also exploring other approaches to further optimize it.

[nikw3f]

Sure let me connect with him

[sinzii]

Hi everyone, please expect the M1 delivery to be delayed a few weeks because we're currently joining the Polkadot Hackathon Vietnam 2023. The project is in a good progress, we're now wrapping up the last pieces of M1, you can check it out now here the repo. Thank you for your understanding!

[takahser]

Hi @sinzii
That's not a problem at all. Could you amend the timeline accordingly?

[sinzii]

@takahser Sure, I've just added a PR for the amendment. Thanks!