Amend Polkadart

[Lohann]

This amend adds a new milestone and update the value of the milestone 3, we underestimated the effort required to support signed transactions, I detailed the new challenges below:

1. Lack of documentation and inconsistencies

Many important components necessary for implement signed extrinsics are not documented or have ambiguous implementations, one example is Substrate BIP-39, while the original Bitcoin BIP39 is well documented, substrate's BIP39 have no documentation at all, so we must rely on the code as reference, it is also fragmented between substrate-bip39 and Substrate repos, and there's a lot of complexity and tiny details there, such as derivation is algorithm dependent, the usage of specific strings for key derivation. Another issue is that the implementation between polkadot-api and subkey are not consistent, example:

import { Keyring } from '@polkadot/keyring';

const keyring = new Keyring({ type: 'sr25519', ss58Format: 42 });
const account = keyring.createFromUri("//18446744073709551616");
console.log(account.address);

// Output: 5FsxbdTHtNSLE5HCLzzGVB2oEzxZB6QJuakiHSoTNWEXD1qj

Now using Substrate's Subkey:

$ subkey inspect --scheme=Sr25519 --network=substrate "//18446744073709551616"

Secret Key URI `//18446744073709551616` is account:
  Network ID:        substrate 
 Secret seed:       0xe95232125504305f665b8e8c891e1e1e87fcbb0b4eff38a9e100862207d2ce97
  Public key (hex):  0x11ea83f6705f1e3bd3b50695227a1c64dccd5c368648f17eb80f093695028daa
  Account ID:        0x11ea83f6705f1e3bd3b50695227a1c64dccd5c368648f17eb80f093695028daa
  Public key (SS58): 5CUCLVQzLh5oxNanu3699rqaxmkiQG4kNNd4WCDfJWiqwuYk
  SS58 Address:      5CUCLVQzLh5oxNanu3699rqaxmkiQG4kNNd4WCDfJWiqwuYk

Both generate different addresses when the junction have more than 64bits. I believe we are the only solution in the ecosystem besides Substrate that correctly implements the Substrate-BIP39 standard:
https://github.com/rankanizer/polkadart/tree/main/packages/substrate_bip39

We found some complexity we didn't anticipate implementing other components too:

• Some signatures are non-standard like sr25519
• UncheckExtrinsics uses a custom encoding/decoding format
• Signed Extensions must be manually implemented on the client-side, etc...

2. No Schnorrkel Library for Dart

Some cryptographic libraries like schnorrkel only exists in Rust, Polkadot-api uses native rust compiled to wasm for signing transactions, we cannot do the same in dart because Flutter runs on web, mobile and desktop devices, we can't assume a native Wasm Runtime like polkadot-api does, and creating native bindings in flutter is not trivial either, it requires us to create one binding for each platform (iOS, Android, Windows, Web, MacOs, etc), and using native bindings requires Flutter developers to have the rust toolchain installed in order to use our library. Another solution is rewrite the schnorrkel in pure dart, we are still deciding how to approach this issue.

3. Extra work for Signed Extrinsics

Substrate metadata itself doesn't provide all necessary info for building a valid transaction, RPC modules and Signed Extensions must be implemented individually for each chain [link], we had to refer to different repos to understand how to properly fill up the signed extensions.

The new Plan

While we have a lot of talented Dart/Flutter developers working in this Project, just a few of them have the necessary experience and understanding of Substrate's codebase to effectively porting it to Dart. Also I would like to highlight that Polkadart is as complex as polkadot-api and capi.
Below the new plan and current development status, we decide to split the milestone 3 in two parts, where on milestone 4 we will focus on Sr25519 and other minor functionalities.

Milestone 3:

• Substrate BIP39
• Ed25519
  • Signatures
  • Hard Junction Derivation
• Ecdsa/secp256k1
  • Signatures
  • Hard Junction Derivation
• Substrate Signed Extensions
  • CheckSpecVersion
  • CheckTxVersion
  • CheckGenesis
  • CheckEra (WIP)
  • CheckNonce
  • CheckWeigh
  • ChargeTransactionPayment
• Statemint Signed Extensions
  • ChargeAssetTxPayment
• Keyring
• Transaction and events subscriptions

Milestone 4:

• Support Sr25519 by rewrite it in Dart or using Native Bindings.
• Support the new Substrate's JSON-RPC API Mega
• Implement's Substrate's Multisig Accounts
• Support custom signed extensions (for non-standard extensions)
• Support custom RPC methods (for non-standard RPC modules)
• Secure Keystore:
  • Keychain for iOS and MacOS
  • Keystore for Android
  • Libsecret for Linux
  • Wincred for Windows
  • WebCrypto for web

We are very proud of Polkadart, even without signed extrinsics, we already have external contributors and the project is being used in production already. Thanks to Polkadart many Flutter developers joined the Substrate ecosystem. We would love to keep supporting this project, count on us to get this done!

[CLAassistant]

All committers have signed the CLA.

[Noc2]

Thanks for creating the amendment and the detailed explanation here. I'm generally happy to go ahead with it, but I have two questions/comments:

• Why did you change the FTE, and why did the cost per developer per month for the fourth milestone increase so much?
• What's your cryptographic background/experience? I think in almost all cases it's better to use bindings for schnorrkel. Otherwise, it might also be necessary to audit the dart implementation.

[semuelle]

pinging @Lohann

[semuelle]

I am going to close this due to inactivity. Please let me know if I should reopen it, @Lohann.