add inscription_indexer.md

[jyaymusk]

Project Abstract

Please replace these instructions with a brief description of your project summarising key points (1-2 paragraphs).

If your application is a follow-up to a previous grant, please mention which one in the first line of the abstract and include a link to previous pull requests if applicable.

Grant level

• Level 1: Up to $10,000, 2 approvals
• Level 2: Up to $30,000, 3 approvals
• Level 3: Unlimited, 5 approvals (for >$100k: Web3 Foundation Council approval)

Application Checklist

• The application template has been copied and aptly renamed (project_name.md).
• I have read the application guidelines.
• Payment details have been provided (Polkadot AssetHub (DOT, USDC & USDT) address in the application and bank details via email, if applicable).
• I am aware that, in order to receive a grant, I (and the entity I represent) have to successfully complete a KYC/KYB check.
• The software delivered for this grant will be released under an open-source license specified in the application.
• The initial PR contains only one commit (squash and force-push if needed).
• The grant will only be announced once the first milestone has been accepted (see the announcement guidelines).
• I prefer the discussion of this application to take place in a private Element/Matrix channel. My username is: @_______:matrix.org (change the homeserver if you use a different one)

[github-actions]

CLA Assistant Lite bot All contributors have signed the CLA ✍️ ✅

[jyaymusk]

I have read and hereby sign the Contributor License Agreement.

[PieWol]

Hey @jyaymusk ,
thank you and your team for the application.

As you correctly mentioned in "Ecosystem Fit" there are other teams who had come up with this idea already a long time ago. I'm looking at RMRK here. As I see it the RMRK team has even deprecated this way of minting by now and is using other ways. On Polkadot / Kusama they now use the uniques pallet for NFT creation and interaction.

I think it would make sense for you to research if there really is a need for this when the history so far shows that it has been successfully delivered and published but still got discontinued. Just for the sake of creating this solution with a different tech stack most likely won't appeal to the grants committee that ultimately decides on applications.

I would suggest to build on top of existing open source solutions in any case. See here the different specs by the RMRK ream.

To now talk about your plans for an indexer, I think there is already an indexer well capable of the needs you describe. I recommend to check out Subsquid and re-evaluate your wish of creating something completely from the ground up.

Let me know what you think.

[jyaymusk]

Hey @jyaymusk , thank you and your team for the application.

As you correctly mentioned in "Ecosystem Fit" there are other teams who had come up with this idea already a long time ago. I'm looking at RMRK here. As I see it the RMRK team has even deprecated this way of minting by now and is using other ways. On Polkadot / Kusama they now use the uniques pallet for NFT creation and interaction.

I think it would make sense for you to research if there really is a need for this when the history so far shows that it has been successfully delivered and published but still got discontinued. Just for the sake of creating this solution with a different tech stack most likely won't appeal to the grants committee that ultimately decides on applications.

I would suggest to build on top of existing open source solutions in any case. See here the different specs by the RMRK ream.

To now talk about your plans for an indexer, I think there is already an indexer well capable of the needs you describe. I recommend to check out Subsquid and re-evaluate your wish of creating something completely from the ground up.

Let me know what you think.

Thank you @PieWol , Thank you very much for your thorough response. We understand that no product can fully meet all users' needs, and we also appreciate the work done by Subsquid, which is truly remarkable and suitable for larger-scale applications. We understand that the scope of data parsing in Inscriptions is not broad, but rather, the key lies in determining the scope, data structures, and rules for data entry into the database. In other words, the challenge lies not in data parsing, but in establishing a standardized approach.

We are focused on how to quickly create products on Polkadot similar to BRC20, or even BRC404. From these perspectives, DOTA has been successful in meeting the needs of these users. If you pay attention to the expectations of our community and the efforts of our team, I believe you will share the same thoughts. Inscriptions Index pays even more attention to the rules of ledgers and tokens, and how related data is entered into the database according to these rules, as well as how to ensure the consistency of community data – all of which are crucial for consensus. Everyone has data stored locally in MySQL, so how to quickly verify that the ledger is consistent with the entire network, and so on, are still many details that need to be addressed.

Currently, we have established the dot-20 specification for indexing in DOTA. This allows developers to create products based on this protocol, similar to ERC20-based products like OpenSea, without needing to rely on smart contracts. To achieve this, the indexer needs to parse and operate on data according to the requirements of dot-20. This requires a new indexer that is dedicated to this direction. We know that off-chain products are more likely to meet user experience expectations, which is a direction worth exploring.

We hope that the creation of this indexer will not only benefit DOTA but also be detached and become a new Inscriptions Indexer specification and a public infrastructure for Polkadot, where anyone can contribute interesting ideas or pull requests, eliminating the need for redundant development. A unified standard will also facilitate information exchange between Inscriptions platforms. Inscriptions is not a last resort; it is a new way of thinking that may drive entirely new interactions between on-chain and off-chain, as well as potentially bringing new technical solutions and interesting products. We should prepare for this. In fact, almost all public chains are doing similar things. A unified specification for the Inscriptions Indexer will also bring the benefit of many people participating in collective construction, ensuring security, and making integration with traditional CEX more efficient and easier to adopt.

[jyaymusk]

Hey @jyaymusk , thank you and your team for the application.
As you correctly mentioned in "Ecosystem Fit" there are other teams who had come up with this idea already a long time ago. I'm looking at RMRK here. As I see it the RMRK team has even deprecated this way of minting by now and is using other ways. On Polkadot / Kusama they now use the uniques pallet for NFT creation and interaction.
I think it would make sense for you to research if there really is a need for this when the history so far shows that it has been successfully delivered and published but still got discontinued. Just for the sake of creating this solution with a different tech stack most likely won't appeal to the grants committee that ultimately decides on applications.
I would suggest to build on top of existing open source solutions in any case. See here the different specs by the RMRK ream.
To now talk about your plans for an indexer, I think there is already an indexer well capable of the needs you describe. I recommend to check out Subsquid and re-evaluate your wish of creating something completely from the ground up.
Let me know what you think.

Thank you @PieWol , Thank you very much for your thorough response. We understand that no product can fully meet all users' needs, and we also appreciate the work done by Subsquid, which is truly remarkable and suitable for larger-scale applications. We understand that the scope of data parsing in Inscriptions is not broad, but rather, the key lies in determining the scope, data structures, and rules for data entry into the database. In other words, the challenge lies not in data parsing, but in establishing a standardized approach.

We are focused on how to quickly create products on Polkadot similar to BRC20, or even BRC404. From these perspectives, DOTA has been successful in meeting the needs of these users. If you pay attention to the expectations of our community and the efforts of our team, I believe you will share the same thoughts. Inscriptions Index pays even more attention to the rules of ledgers and tokens, and how related data is entered into the database according to these rules, as well as how to ensure the consistency of community data – all of which are crucial for consensus. Everyone has data stored locally in MySQL, so how to quickly verify that the ledger is consistent with the entire network, and so on, are still many details that need to be addressed.

Currently, we have established the dot-20 specification for indexing in DOTA. This allows developers to create products based on this protocol, similar to ERC20-based products like OpenSea, without needing to rely on smart contracts. To achieve this, the indexer needs to parse and operate on data according to the requirements of dot-20. This requires a new indexer that is dedicated to this direction. We know that off-chain products are more likely to meet user experience expectations, which is a direction worth exploring.

We hope that the creation of this indexer will not only benefit DOTA but also be detached and become a new Inscriptions Indexer specification and a public infrastructure for Polkadot, where anyone can contribute interesting ideas or pull requests, eliminating the need for redundant development. A unified standard will also facilitate information exchange between Inscriptions platforms. Inscriptions is not a last resort; it is a new way of thinking that may drive entirely new interactions between on-chain and off-chain, as well as potentially bringing new technical solutions and interesting products. We should prepare for this. In fact, almost all public chains are doing similar things. A unified specification for the Inscriptions Indexer will also bring the benefit of many people participating in collective construction, ensuring security, and making integration with traditional CEX more efficient and easier to adopt.

Nearly 50 people are currently involved in the construction of this indexer. Due to the fact that the dota product is still in its early stages and requires absolute security, we only allow them to participate in testing and propose ideas.

[jyaymusk]

Hey @jyaymusk , thank you and your team for the application.
As you correctly mentioned in "Ecosystem Fit" there are other teams who had come up with this idea already a long time ago. I'm looking at RMRK here. As I see it the RMRK team has even deprecated this way of minting by now and is using other ways. On Polkadot / Kusama they now use the uniques pallet for NFT creation and interaction.
I think it would make sense for you to research if there really is a need for this when the history so far shows that it has been successfully delivered and published but still got discontinued. Just for the sake of creating this solution with a different tech stack most likely won't appeal to the grants committee that ultimately decides on applications.
I would suggest to build on top of existing open source solutions in any case. See here the different specs by the RMRK ream.
To now talk about your plans for an indexer, I think there is already an indexer well capable of the needs you describe. I recommend to check out Subsquid and re-evaluate your wish of creating something completely from the ground up.
Let me know what you think.

Thank you @PieWol , Thank you very much for your thorough response. We understand that no product can fully meet all users' needs, and we also appreciate the work done by Subsquid, which is truly remarkable and suitable for larger-scale applications. We understand that the scope of data parsing in Inscriptions is not broad, but rather, the key lies in determining the scope, data structures, and rules for data entry into the database. In other words, the challenge lies not in data parsing, but in establishing a standardized approach.

We are focused on how to quickly create products on Polkadot similar to BRC20, or even BRC404. From these perspectives, DOTA has been successful in meeting the needs of these users. If you pay attention to the expectations of our community and the efforts of our team, I believe you will share the same thoughts. Inscriptions Index pays even more attention to the rules of ledgers and tokens, and how related data is entered into the database according to these rules, as well as how to ensure the consistency of community data – all of which are crucial for consensus. Everyone has data stored locally in MySQL, so how to quickly verify that the ledger is consistent with the entire network, and so on, are still many details that need to be addressed.

Currently, we have established the dot-20 specification for indexing in DOTA. This allows developers to create products based on this protocol, similar to ERC20-based products like OpenSea, without needing to rely on smart contracts. To achieve this, the indexer needs to parse and operate on data according to the requirements of dot-20. This requires a new indexer that is dedicated to this direction. We know that off-chain products are more likely to meet user experience expectations, which is a direction worth exploring.

We hope that the creation of this indexer will not only benefit DOTA but also be detached and become a new Inscriptions Indexer specification and a public infrastructure for Polkadot, where anyone can contribute interesting ideas or pull requests, eliminating the need for redundant development. A unified standard will also facilitate information exchange between Inscriptions platforms. Inscriptions is not a last resort; it is a new way of thinking that may drive entirely new interactions between on-chain and off-chain, as well as potentially bringing new technical solutions and interesting products. We should prepare for this. In fact, almost all public chains are doing similar things. A unified specification for the Inscriptions Indexer will also bring the benefit of many people participating in collective construction, ensuring security, and making integration with traditional CEX more efficient and easier to adopt.

Our plan is to create a parachain focused on inscriptions, going beyond just interacting with the chain through 'remark_with_event'. We hope that the Inscriptions platform can interact with the chain at a system level simultaneously. For instance, besides sending 'remark_with_event', we aim to enable the Inscriptions platform to send calldata or other types of memos to the chain. These calldata or memos can trigger operations on the chain, allowing for deeper interaction between the chain and the Inscriptions platform. The foundation for realizing this vision lies in having a certain type of Inscriptions indexer and rules. This is very appealing as it can meet the needs of different users. One can imagine that in the future, operations that would typically require a significant amount of gas fees could be executed with just a simple memo, saving a lot of costs for users. Operations that would typically occur on-chain, such as in social and gaming contexts, could be combined on-chain and off-chain, greatly optimizing the user experience, among other benefits.

[jyaymusk]

Hey @jyaymusk , thank you and your team for the application.
As you correctly mentioned in "Ecosystem Fit" there are other teams who had come up with this idea already a long time ago. I'm looking at RMRK here. As I see it the RMRK team has even deprecated this way of minting by now and is using other ways. On Polkadot / Kusama they now use the uniques pallet for NFT creation and interaction.
I think it would make sense for you to research if there really is a need for this when the history so far shows that it has been successfully delivered and published but still got discontinued. Just for the sake of creating this solution with a different tech stack most likely won't appeal to the grants committee that ultimately decides on applications.
I would suggest to build on top of existing open source solutions in any case. See here the different specs by the RMRK ream.
To now talk about your plans for an indexer, I think there is already an indexer well capable of the needs you describe. I recommend to check out Subsquid and re-evaluate your wish of creating something completely from the ground up.
Let me know what you think.

Thank you @PieWol , Thank you very much for your thorough response. We understand that no product can fully meet all users' needs, and we also appreciate the work done by Subsquid, which is truly remarkable and suitable for larger-scale applications. We understand that the scope of data parsing in Inscriptions is not broad, but rather, the key lies in determining the scope, data structures, and rules for data entry into the database. In other words, the challenge lies not in data parsing, but in establishing a standardized approach.
We are focused on how to quickly create products on Polkadot similar to BRC20, or even BRC404. From these perspectives, DOTA has been successful in meeting the needs of these users. If you pay attention to the expectations of our community and the efforts of our team, I believe you will share the same thoughts. Inscriptions Index pays even more attention to the rules of ledgers and tokens, and how related data is entered into the database according to these rules, as well as how to ensure the consistency of community data – all of which are crucial for consensus. Everyone has data stored locally in MySQL, so how to quickly verify that the ledger is consistent with the entire network, and so on, are still many details that need to be addressed.
Currently, we have established the dot-20 specification for indexing in DOTA. This allows developers to create products based on this protocol, similar to ERC20-based products like OpenSea, without needing to rely on smart contracts. To achieve this, the indexer needs to parse and operate on data according to the requirements of dot-20. This requires a new indexer that is dedicated to this direction. We know that off-chain products are more likely to meet user experience expectations, which is a direction worth exploring.
We hope that the creation of this indexer will not only benefit DOTA but also be detached and become a new Inscriptions Indexer specification and a public infrastructure for Polkadot, where anyone can contribute interesting ideas or pull requests, eliminating the need for redundant development. A unified standard will also facilitate information exchange between Inscriptions platforms. Inscriptions is not a last resort; it is a new way of thinking that may drive entirely new interactions between on-chain and off-chain, as well as potentially bringing new technical solutions and interesting products. We should prepare for this. In fact, almost all public chains are doing similar things. A unified specification for the Inscriptions Indexer will also bring the benefit of many people participating in collective construction, ensuring security, and making integration with traditional CEX more efficient and easier to adopt.

Our plan is to create a parachain focused on inscriptions, going beyond just interacting with the chain through 'remark_with_event'. We hope that the Inscriptions platform can interact with the chain at a system level simultaneously. For instance, besides sending 'remark_with_event', we aim to enable the Inscriptions platform to send calldata or other types of memos to the chain. These calldata or memos can trigger operations on the chain, allowing for deeper interaction between the chain and the Inscriptions platform. The foundation for realizing this vision lies in having a certain type of Inscriptions indexer and rules. This is very appealing as it can meet the needs of different users. One can imagine that in the future, operations that would typically require a significant amount of gas fees could be executed with just a simple memo, saving a lot of costs for users. Operations that would typically occur on-chain, such as in social and gaming contexts, could be combined on-chain and off-chain, greatly optimizing the user experience, among other benefits.

We acknowledge the pioneering work done by the RMRK team in the early stages, which was truly ahead of its time. However, they clearly did not anticipate that inscriptions would become such a hot trend in the blockchain market and serve as a complement to blockchain technology, as evidenced by the current trend across all public chains. Now, there is much to be done, extending beyond just NFTs and branching out into various other scenarios. This includes adhering to the same protocols as other public chains' inscriptions, innovating new protocols, and following the direction of the latest inscription products.

We aim to establish a reliable and secure index on DOTA, allowing everyone to use it with confidence and enabling more applications to be built on top of this indexer. This indexer not only parses transactions and stores them in MySQL but, more importantly, filters data based on protocol rules, executes rules (protocols), and aggregates data. This also means that we are not simply a web scraper for native chain data.

While we hope that Subsquid's technology can assist us in this endeavor, it is not currently necessary. We have a team of seasoned engineers familiar with Substrate, with five years of experience in the Polkadot ecosystem. In simple terms, what DOTA is doing is not in conflict with what other projects in the ecosystem are doing; rather, it complements them and innovates based on the excellent ideas of these projects. The unprecedented transaction volume during the DOTA minting period on the Polkadot chain is evidence enough of the market demand and the need for deeper technical exploration within the Polkadot ecosystem in this field.

The technical exploration in the field of inscriptions should not stagnate in the past.

[PieWol]

Hey @jyaymusk,
please refrain from quoting your answers while still answering another post. It just creates this huge wall of text which is unpleasant to read.

I read this post on the polkadot forum. Have you tried to get funding from the treasury too? What was the outcome?

You say that subsquid is not suitable for your needs and you need to develop something yourself. Can you please concisely list what exactly subsquid is missing?
Thank you.

[PieWol]

Hey @jyaymusk hope you are doing well. Do you have any updates to share?

[jyaymusk]

Hey @jyaymusk hope you are doing well. Do you have any updates to share?

Thank you very much for your patient response and cautious attitude towards the Polkadot grant funding. Indeed, over the past month, we have initiated a proposal for Polkadot Treasury funds, and we are very grateful for the extensive participation in this discussion, which was beyond our expectations. This is also one of the few Treasury proposals in Polkadot's history to have such extensive participation and involvement of DOT holders.

We regret that we were unable to obtain funding, but this will not stop us from continuing to build our ecosystem and products. We have made significant progress in engaging and communicating with the community and are confident that we will soon launch another proposal. We are confident that we will gain understanding and support. Thank you!

[jyaymusk]

Hey @jyaymusk hope you are doing well. Do you have any updates to share?

This project is an indexer for dota. It has its own protocol to comply with, such as dot20, a protocol like ERC20, which also determines the specificity of the function of the indexer itself. This new indexer is now in testing and entering its final stages. Its completion means a comprehensive upgrade of Dota and more community products will be built on it, which can directly help the community. We also know that subsquid can obtain on-chain data, but it is different from our direction. We focus more on the development of indexers based on specific protocols, and data acquisition and mining based on protocols. This is the code for our indexer to obtain and filter data on the chain, and dot-20 code. If you have time, you can take a look and provide suggestions. It may also be helpful to our further discussions.

[semuelle]

recheck

[semuelle]

Thanks for the application and sorry for the long wait, @jyaymusk. Please see my questions below.

[semuelle]

What are "user tick balances" and why do they need storing in a MySQL database?

[jyaymusk]

What are "user tick balances" and why do they need storing in a MySQL database?

It is the balance of the user's token. In Inscription, we prefer to call it tick. Because in MySQL, it brings difficulties to reconciliation, it needs to be stored in the Merkle tree structure. When different indexers need to know whether they are the same, they only need to compare the Merkle roots.

[semuelle]

Isn't this already largely done?

[jyaymusk]

Isn't this already largely done?

Yes, we have completed most of the discussions and modifications to the dot20 protocol. And the relevant codes are constantly being updated and improved.

[semuelle]

Standards are usually written by committees with a broad range of stakeholders. Can you share who was part of the process and how it came about?

[jyaymusk]

Standards are usually written by committees with a broad range of stakeholders. Can you share who was part of the process and how it came about?

The creation of dot20 is a very interesting thing. It is based on extensive community discussions and was summarized and written by our dota indexer protocol team. We have changed many versions, but are not satisfied with them. The reason is that in the process of creating the marketplace, the community cannot develop products as decentralized (or as decentralized) as opensea, so you see dota’s marketplace It has not been developed yet. Fortunately, the community found a protocol standard suitable for it during the discussion, which is the dot20 protocol

[semuelle]

You mentioned in one of your comments that the team has "five years of experience in the Polkadot ecosystem". However, I don't see much activity on the Github profiles you linked. Could you elaborate?

[jyaymusk]

Thank you very much to substrate for bringing us such a wonderful development experience. Personally, I have joined the development of substrate a long time ago and have my own project. It was 2019. The founder of the acala team introduced Polkadot technology to China and provided guidance and education on community technology. I was lucky to be the first beneficiary. Obviously, I changed my github account because different teams have different requirements for privacy and need us to keep our past experiences confidential.

[semuelle]

Thanks for the updates, @jyaymusk. Given the huge amount of ways to create assets on Polkadot, I don't think another standard is necessary. If you can show that a sufficiently large part of the community thinks otherwise, please let me know. Until then, I am not going to support this application. However, I will notify the rest of the committee, who might disagree with me.

[Noc2]

Thanks a lot for the application and all the effort you put into this. However, the grants committee decided not to go ahead with it, mostly because of the reasons mentioned here before. Independent of this, we wish you all the best for your project. Let us know if you have any questions.

[Noc2]

see above