feat: rework test-harness for integration tests

[lima-limon-inc]

Tackles #876

This PR aims to re work the testing harness merged here in order for it to work for integration tests.

A thing that was common in all the integration-network tests was the presence of "repetitive" boilerplate code to set up a couple of basic wallet accounts inside a MockChain. See:

• test_basic_wallet_p2id
• test_basic_wallet_p2ide
• test_basic_wallet_p2ide_reclaim

With this in mind, this PR aims to aid in reducing the amount of boilerplate required to set up the context for a test. To achieve this, I went with a clap-esque/operation-esque approach.

Currently, it looks like so:

#[miden_test(
    chain(name = "builder"),
    faucet(name = "faucet", max_supply = 1_000_000_000)
)]
pub fn test_basic_wallet_p2id() {
    (...)
}

Which replaced the following lines:

#[test]
pub fn test_basic_wallet_p2id() {
    (...)
    let mut builder = MockChain::builder();
    let max_supply = 1_000_000_000u64;
    let faucet_account = builder
        .add_existing_basic_faucet(Auth::BasicAuth, "TEST", max_supply, None)
        .unwrap();
    (...)        
}

The plan is to have something along these lines:

#[miden_test(
    chain(name = "builder"),
    faucet(name = "faucet", max_supply = 1_000_000_000),
    account(name = "bob"),
    account(name = "alice"),
)]
pub fn test_basic_wallet_p2id() {
    (...)
}

To replace the following:

#[test]
pub fn test_basic_wallet_p2id() {
    let mut builder = MockChain::builder();
    let max_supply = 1_000_000_000u64;
    let faucet_account = builder
        .add_existing_basic_faucet(Auth::BasicAuth, "TEST", max_supply, None)
        .unwrap();

    let alice_account = builder
        .add_account_from_builder(
            Auth::BasicAuth,
            build_existing_basic_wallet_account_builder(wallet_package.clone(), false, [1_u8; 32]),
            AccountState::Exists,
        )
        .unwrap();

    let bob_account = builder
        .add_account_from_builder(
            Auth::BasicAuth,
            build_existing_basic_wallet_account_builder(wallet_package, false, [2_u8; 32]),
            AccountState::Exists,
        )
        .unwrap();
}

This probably supersede the previous syntax where the "special"/"magic" recognized keywords were passed as function arguments. I believe this attribute interface is more idiomatic than the current interface.

Concretely, this means going from this:

    #[miden_test]
    fn foo(chain: MockChainBuilder) {
        assert_eq!(2, 1 + 1)
    }

to:

    #[miden_test(
    chain(name = "chain")
    )]
    fn foo(chain: MockChainBuilder) {
        assert_eq!(2, 1 + 1)
    }

This PR is still in a WIP state and can of course include even more functionality (for instance, creating n instances of an Account`).

Implementation wise, the attributes follow a "pipeline" in order to emit the code:

1. Parsed with darling into a XBuilder struct
2. These XBuilder structs are validated, via their own validate function.
3. After validation, each XBuilder struct gets turned into a plain X
4. (Currently not implemented): A "dependency resolution" step which would order the structs in the required order. (e.g. the "faucet"'s generated code requires the "MockChain"'s code to be generated it before it).
5. Finally, each structs emits their corresponding code.

This pipeline can be found here.

[lima-limon-inc]

@greenhat @bitwalker The PR is not yet finished, but what do you think about this approach?

Any comments / recommendations are more than welcome.

Thanks in advance!

[greenhat]

Good job!

The plan is to have something along these lines:

#[miden_test(
    chain(name = "builder"),
    faucet(name = "faucet", max_supply = 1_000_000_000),
    account(name = "bob"),
    account(name = "alice"),
)]
pub fn test_basic_wallet_p2id() {
    (...)
}

The only problem with attribute macro DSLs I see is discoverability by the user. To this day I have to jump and search the docs when I'm using clap. OTOH, it's powerful and we can design it however we want. Let's try to implement the example above and see how it goes.

[lima-limon-inc]

Good job!

Thank you very much! 😊

The only problem with attribute macro DSLs I see is discoverability by the user. To this day I have to jump and search the docs when I'm using clap.

Completely agree, clap can definitely feel like a maze at times. I'll keep discoverability in mind for this PR.

The first thing that comes to mind is implementing a "help" command in the macro itself. Something like:

#[miden_test(help)]
pub fn test_basic_wallet_p2id() {
    (...)
}

Which would display all of the supported attributes.

Whilst I haven't seen this idea of a "help" command in macros, it's pretty standard in CLI tools; so maybe the it isn't too far out there.

I'll tackle it and see how it goes. Do let me know if you have any thoughts!

[lima-limon-inc]

#[miden_test(help)]
pub fn test_basic_wallet_p2id() {
    (...)
}

Which would display all of the supported attributes.

Whilst I haven't seen this idea of a "help" command in macros, it's pretty standard in CLI tools; so maybe the it isn't too far out there.

I'll tackle it and see how it goes. Do let me know if you have any thoughts!

I have just pushed the code that supports #[miden_test(help)]. When used, it will produce the following output:

--> tests/integration-network/src/mockchain/basic_wallet.rs:20:1
 |
 | #[miden_test(help)]
 | ^^^^^^^^^^^^^^^^^^^
 |
 = help: message:
   <documentation string>

Where <documentation string> is the documentation for every supported attribute (and its corresponding recognized fields). The list of supported struct is none other than all the enum variants present in RecognizedAttrsBuilder.

A nifty detail is that it uses each struct's doc-comments to generate the help message. Which means that it remains consistent and up to date with the generated cargo-doc webpage.

Additionally, I also added support for #[miden_test(help(attribute))] which only displays <attribute>'s help message.

The actual displayed help message is still a bit rough around the edges, but all the "logic" to collect and generate the message is all there.

I'll update the PR as soon as that is done.

Just wanted to update the PR to show how things are shaping up 😊. Any comment/suggestion is more than welcome!

Cheers!

[greenhat]

I have just pushed the code that supports #[miden_test(help)]. When used, it will produce the following output:

--> tests/integration-network/src/mockchain/basic_wallet.rs:20:1
 |
 | #[miden_test(help)]
 | ^^^^^^^^^^^^^^^^^^^
 |
 = help: message:
   <documentation string>

Where <documentation string> is the documentation for every supported attribute (and its corresponding recognized fields). The list of supported struct is none other than all the enum variants present in RecognizedAttrsBuilder.

A nifty detail is that it uses each struct's doc-comments to generate the help message. Which means that it remains consistent and up to date with the generated cargo-doc webpage.

Nice! I have not thought about calling a macro to get help.

[lima-limon-inc]

@bitwalker @greenhat I believe the PR is ready for review. Do note that it mainly focuses on refactoring the present test harness infrastructure itself to make it compatible with integration tests.
The main changes are the introduction of the #[miden_test] attribute instead of the previous approach where the "magic" miden-testing structures were generated from passed in arguments.
This new macros currently support:

• account
• chain
• faucet
• package
• help: this is a special attribute which will display all the available documentation. This is generated from the doc comments from all the various structs and its fields.

The actual expansion of the present tests could probably be done in a separate PR since this already is quite extensive.

Would love to hear your thoughts!

PS: The CI is currently running but it should pass, I've just pushed a commit updating the documentation.

[bitwalker]

Definitely cleans up the boilerplate!

I do have some reservations about the level of magic involved, but the help attribute does mitigate that for the most part (of course, one has to know to use that in the first place, but it's still better than nothing). I definitely wish that attribute macros supported the same "helper" attributes mechanism that derive macros do, since that would help with discoverability, but 🤷.

I would suggest we provide a higher-level "user guide"-type document as well, maybe in test-harness/README.md, which goes into detail how the different options stack/interact, what the defaults are, and whether some options require others to be provided, and if so, under what conditions. That sort of guidance is the main thing missing IMO, as currently we largely are just relying just on a small set of examples, and the minimal help documentation produced by the help argument (which isn't very discoverable/accessible unless you look at the examples where it is featured).

I'm marking this approved though, and we can merge this once an initial version of that document is added here, just ping me!

[lima-limon-inc]

Definitely cleans up the boilerplate!

I'm glad it helps 😊

I would suggest we provide a higher-level "user guide"-type document as well, maybe in test-harness/README.md, which goes into detail how the different options stack/interact, what the defaults are, and whether some options require others to be provided, and if so, under what conditions. That sort of guidance is the main thing missing IMO, as currently we largely are just relying just on a small set of examples, and the minimal help documentation produced by the help argument (which isn't very discoverable/accessible unless you look at the examples where it is
featured).

I'm marking this approved though, and we can merge this once an initial version of that document is added here, just ping me!

@bitwalker
Noted, I've just pushed b4f6844 which contains a README.md containing said documentation. Was something like this what you had in mind?
Thanks in advance!

[greenhat]

Good job! Very thorough exploration of what's possible to achieve with macros.

I think that the macro parameters approach in general cost/benefit is not looking good.

Let's look at the following macro parameters:

   account(
        name = "alice_account",
        component = "wallet_package",
        seed = 5,
        with_basic_wallet = true
    ),

It is not much more concise than our current ad hoc and unoptimized API:

   let alice_account = builder
        .add_account_from_builder(
            Auth::BasicAuth,
            build_existing_basic_wallet_account_builder(wallet_package.clone(), true, [5_u8; 32]),
            AccountState::Exists,
        );

But it is significantly more expensive for the developers.

I consider the following to be the extra price that the developers would pay for macros on top of Mockchain (which is not going anywhere):

1. Figuring out, understanding and remembering the entirely new for them macro parameters system.
2. No discoverability, no autocomplete, and no parameters help in an IDE.
3. Limited LSP; no refactoring (renaming alice_account or wallet_package would not work), limited "find references" and "goto definition".
4. Deciphering runtime errors encountered in the macros (wrong path, no .masp file, etc.).
5. LLMs are struggling with the unknown macro params system.

With all that in mind I think that we should consider that macros might not be worth pursuing and we might need to change the direction. The alternatives I see:

1. Some hybrid (declarative/imperative) API on top of MockChain. @bitwalker Like the original Scenario but with imperative escape hatches.
2. Improve MockChain API. Make it more concise for default cases.