Wide merkle query #16093

fabrizio-m · 2024-09-19T13:19:58Z

Explain your changes:
Adding support for wide merkle queries that request more than 2 nodes during sync.

Explain how you tested your changes:
Ran V2 node, for this V1 -> V2 case I started the V2, got it to connect to devnet, and then started the V1 with the running V2 as only peer.

Checklist:

Dependency versions are unchanged
- Notify Velocity team if dependencies must change in CI
Modified the current draft of release notes with details on what is completed or incomplete within this project
Document code purpose, how to use it
- Mention expected invariants, implicit constraints
Tests were added for the new behavior
- Document test purpose, significance of failures
- Test names should reflect their purpose
All tests pass (CI will check this if you didn't)
Serialized types are in stable-versioned modules
Does this close issues? List them

mrmr1993 · 2024-09-19T14:41:46Z

src/lib/syncable_ledger/syncable_ledger.ml

+        | Subtree of 'addr
+            (** What are the 2^k nodes at depth k from the given prefix 
+            address **)
+      (* TODO: Properly handle versioning *)
+      (* TODO: Consider additional query to verify subtree suport, for
+         softfork compatibility *)


Unfortunately, we aren't able to modify this type here, we'll have to add a converter. Some nodes won't have upgraded, so we need to be able to have a fallback.

For example, it might be natural to change e.g.

| What_child_hashes of 'addr

into e.g.

| What_child_hashes of {address: 'addr; depth: int}

and then the response can be tweaked from

| Child_hashes_are of 'hash * 'hash

to

| Child_hashes_are of 'hash array

I think this gives a natural downgrade path

function | What_child_hashes {address; depth = _} -> V1.What_child_hashes address | ...

and upgrade for the response of

function | Child_hashes_are (x, y) -> V2.Child_hashes_are [x; y] | ...

As currently written, a node running this code won't be able to sync to the network :'(

To be sure I was thinking the same, for now I added it in V1 just to make it compile until I understand better how versioning works, but I agree it should fail right now.
But if I add a V2 I can add another variant or make arbitrary changes to the type, right? or are there additional limitations when making a new version? and for that reason you suggest modifying Child_hashes_are instead?

I think I understand the example, but I see a possible issue:
Send V2 request, receiver converts it into V1 and sends back V1 response, that is converted to V2 response after arrival, although a trivial one.
But in the opposite case sending a V1 and receiving a V2 response shouldn't work, as the V1 node can convert V2 back to V1.
Or does the example assume both types have the same representation and a V2 can be deserialized as V1 without knowing anything about V2?

The versioning system should ensure that a V1 request never receives a V2 response. The RPC protocol does version negotiation.

For reference, this is where the version negotiation happens.

svv232 · 2024-09-20T19:35:22Z

src/lib/syncable_ledger/syncable_ledger.ml

+      let to_latest = function
+        | What_child_hashes addr ->
+            What_child_hashes addr
+        | What_contents addr ->
+            What_contents addr
+        | Num_accounts ->
+            Num_accounts


why not just Fn.id ?

Actually, it may not be necessary, as I was ultimately decided to not change the query, and if it worked without having to_latest it should still work without it, I'll remove it.

mrmr1993 · 2024-09-23T13:35:49Z

src/lib/syncable_ledger/syncable_ledger.ml

+        | Subtree of 'hash list
+            (** The subtree rooted on the requested address has these leaves *)


Why add this instead of modifying Child_hashes_are? As written, now we have two responses with overlapping behaviour.

Nit: array is usually better, especially where we have a fixed data size that we won't be changing (like here).

I think there is no reason anymore, I'll unify on Child_hashes_are.

mrmr1993 · 2024-09-23T14:00:46Z

src/lib/syncable_ledger/syncable_ledger.ml

+        | Subtree of 'addr
+            (** What are the 2^k nodes at depth k from the given prefix 
+            address **)
+      (* TODO: Properly handle versioning *)
+      (* TODO: Consider additional query to verify subtree suport, for
+         softfork compatibility *)


For reference, this is where the version negotiation happens.

fabrizio-m · 2024-09-23T16:20:53Z

!ci-build-me

fabrizio-m · 2024-09-23T17:30:49Z

!ci-build-me

fabrizio-m · 2024-09-23T17:45:59Z

!ci-build-me

svv232 · 2024-09-20T19:38:51Z

src/lib/syncable_ledger/syncable_ledger.ml

+  (* TODO: parameterize *)
+  let subtree_depth : index = 4
+
+  (* Provides addresses at an specific depth from this address *)


Provides addresses at a specific depth from this address

Per this comment, nit:

Suggested change

(* Provides addresses at an specific depth from this address *)

(* Provides addresses at a specific depth from this address *)

svv232 · 2024-09-23T18:47:30Z

src/lib/syncable_ledger/syncable_ledger.ml

+            (** The requested address's children have these hashes.
+            May be any power of 2 number of children, and not necessarily 
+            immediate children  *)


svv232 · 2024-09-23T18:58:39Z

src/lib/syncable_ledger/syncable_ledger.ml

-      expect_children t addr exp_hash ;
-      Linear_pipe.write_without_pushback_if_open t.queries
-        (desired_root_exn t, What_child_hashes addr) )
+    else expect_children t addr exp_hash ;


you should be calling expect subtree here

maybe this should just be the old version as discussed

svv232 · 2024-09-23T19:02:13Z

src/lib/syncable_ledger/syncable_ledger.ml

+  let expect_subtree : 'a t -> Addr.t -> Hash.t -> unit =
+   fun t parent_addr expected ->
+    [%log' trace t.logger]
+      ~metadata:
+        [ ("subtree prefix address", Addr.to_yojson parent_addr)
+        ; ("hash", Hash.to_yojson expected)
+        ]
+      "Expecting subtree at address $parent_address, expected: $hash" ;
+    Addr.Table.add_exn t.waiting_parents ~key:parent_addr ~data:expected


you can remove this, in lieu of expect children?

svv232 · 2024-09-23T19:11:28Z

src/lib/syncable_ledger/syncable_ledger.ml

+         | `Non_power ] =
+   fun t addr nodes ->
+    (* let prefix_depth = Addr.depth addr in *)
+    if Int.is_pow2 (List.length nodes) then


does this work for 2^0

Yes. You could always send back the same hash and leave the node stuck in a loop here. This needs to reject (List.length nodes) < 2.

svv232 · 2024-09-23T19:17:12Z

src/lib/syncable_ledger/syncable_ledger.ml

+
+  let merge_many : Hash.t list -> index -> Hash.t =
+   fun nodes depth ->
+    let final_depth = depth + subtree_depth in


maybe check if the final depth is greater than the original ledger depth.

Or maybe you should check if the final depth is < ledger depth - children tree depth, which in this case is 6. I am referring to the What Contents are query returning trees with 2^6 children.

also check if responses are 2^4 for the what children are query at some point

svv232 · 2024-09-23T19:26:39Z

src/lib/syncable_ledger/syncable_ledger.ml

+        Option.value_exn ~message:"Forgot to wait for a node"
+          (Addr.Table.find t.waiting_parents addr)
+      in
+      let merged = merge_many nodes (ledger_depth - Addr.depth addr) in


this is really clever bravo! the idea being you only get back leaves and can recompute the intermediary hashes by computing parent siblings. This way you encode more information over the network and require less total network bandwidth.

mrmr1993

This doesn't compile, and isn't integrated with the networking layer. Please fix it.

mrmr1993 · 2024-09-24T11:13:39Z

src/lib/syncable_ledger/syncable_ledger.ml

+         | `Non_power ] =
+   fun t addr nodes ->
+    (* let prefix_depth = Addr.depth addr in *)
+    if Int.is_pow2 (List.length nodes) then


Yes. You could always send back the same hash and leave the node stuck in a loop here. This needs to reject (List.length nodes) < 2.

mrmr1993 · 2024-09-24T11:16:10Z

src/lib/syncable_ledger/syncable_ledger.ml

+            let ledger_depth = MT.depth mt in
+            let addresses = intermediate_range ledger_depth a subtree_depth in
+            let get_hash a = MT.get_inner_hash_at_addr_exn mt a in
+            let hashes = List.map addresses ~f:get_hash in


You should use Or_error.try_with to catch and handle any errors, like the previous version of the code did.

mrmr1993 · 2024-09-24T11:17:34Z

src/lib/syncable_ledger/syncable_ledger.ml

@@ -221,6 +235,28 @@ end = struct

  type query = Addr.t Query.t

+  (* TODO: parameterize *)
+  let subtree_depth : index = 4


Agree with the comment, this should be parameterised.

About that, the initial idea was for the request to specify this depth, but as I ended up changing only the response that wouldn't be possible.
It could be set on the responder node but that would potentially allow the responder to send very big responses to make you waste time.
What did you have in mind?

mrmr1993 · 2024-09-24T11:17:44Z

src/lib/syncable_ledger/syncable_ledger.ml

@@ -221,6 +235,28 @@ end = struct

  type query = Addr.t Query.t

+  (* TODO: parameterize *)
+  let subtree_depth : index = 4


The type here is int, not the local index.

mrmr1993 · 2024-09-24T11:21:29Z

src/lib/syncable_ledger/syncable_ledger.ml

+         | `Non_power ] =
+   fun t addr nodes ->
+    (* let prefix_depth = Addr.depth addr in *)
+    if Int.is_pow2 (List.length nodes) then


This condition makes the protocol 'sender decides' instead of 'requester requests'. You should consider the original design I described: the requester sends the depth below the address that they would like, then the sender responds with a depth less than or equal to that, and the requester only accepts the message if the length is less than or equal.

The issue with changing the request is that then V2 nodes wouldn't be able to connect to V1 nodes to sync, and no synced V2 node would come into existence to allow other V2s to sync, which wouldn't be softfork compatible.

This is mediated at the network layer, as I had already mentioned here.

mrmr1993 · 2024-09-24T11:25:35Z

src/lib/syncable_ledger/syncable_ledger.ml

+        | What_child_hashes a ->
+            let ledger_depth = MT.depth mt in
+            let addresses = intermediate_range ledger_depth a subtree_depth in
+            let get_hash a = MT.get_inner_hash_at_addr_exn mt a in


When this hits the ledger DB, looping over these one at a time will be slow. You should use one of the batch functions instead.

Which function are you thinking about?
I considered that but there doesn't seem to be functions for arbitrary batch reads.
There is get_all_accounts_rooted_at_exn which provides subtrees but not with variable depth, and internally seems to be looping over a list and reading each entry individually.

It seems that the underlying Ledger.get_hash_batch isn't exposed. I suppose this can wait for later then.

mrmr1993 · 2024-09-24T11:29:42Z

src/lib/syncable_ledger/syncable_ledger.ml

+         | `Non_power ] =
+   fun t addr nodes ->
+    (* let prefix_depth = Addr.depth addr in *)
+    if Int.is_pow2 (List.length nodes) then


List.length is significantly more expensive than Array.length. Please use array above, as requested before

mrmr1993 · 2024-09-24T11:36:37Z

src/lib/syncable_ledger/syncable_ledger.ml

+                          ( "hashes sent for subtree on address $address merge \
+                             to $actualmerge but we expected $expectedmerge"
+                          , [ ("actualmerge", Hash.to_yojson actual)
+                            ; ("expectedmerge", Hash.to_yojson expected)


Prefer snake-case: $actual_merge, $expected_merge.

mrmr1993 · 2024-09-24T11:38:28Z

src/lib/syncable_ledger/syncable_ledger.ml

+        (* TODO: give some error as this shouldn't really happen *)
+        []


This definitely can happen if you receive 1 hash in response. Please handle this properly.

fabrizio-m · 2024-09-24T15:54:19Z

!ci-build-me

fabrizio-m · 2024-09-24T17:25:02Z

!ci-build-me

mrmr1993 · 2024-09-25T08:32:35Z

This PR should be against compatible, please rebase your changes on top of that

fabrizio-m · 2024-09-25T12:42:41Z

!ci-build-me

fabrizio-m · 2024-09-25T17:46:14Z

!ci-build-me

fabrizio-m · 2024-09-25T19:56:55Z

!ci-build-me

fabrizio-m · 2024-09-26T12:03:31Z

!ci-nightly-me

…mon; use block window duration from runtime config and not compile config; remove compile_config from impacted contexts

…-merkle-queries

mrmr1993 · 2024-11-13T00:56:45Z

src/lib/genesis_constants/genesis_constants.ml

@@ -223,6 +223,8 @@ module T = struct
    ; max_action_elements : int
    ; zkapp_cmd_limit_hardcap : int
    ; minimum_user_command_fee : Currency.Fee.Stable.Latest.t
+    ; sync_ledger_default_subtree_depth : int


These really shouldn't be a part of Genesis_constants.

I wasn't sure where it should be. It seems like we are working towards removing compile config. Any suggestion?

Maybe create something new like Daemon_constants that should include a lot of what's in Genesis_constants?

I think the daemon section of the runtime config makes sense, or as a flag. I'd be fine if it followed the same logic as snark_worker_fee.

There are two constants (three if we extracted account_subtree_height later) and they need to be threaded through a few different modules- bootstrap_controller, block_producer and consensus. However, they are used only in syncable_ledger.ml and so exposing it along the way doesn’t seem nice in terms of readability. Grouping and passing them in contexts seemed better.

Moved it back to compile time. I'll propose this in a different PR

src/lib/mina_ledger/sync_ledger.ml

…-merkle-queries

deepthiskumar · 2024-11-19T19:35:55Z

!ci-build-me

deepthiskumar · 2024-11-19T19:36:04Z

!ci-nightly-me

Co-authored-by: Sai Vegasena <[email protected]>

deepthiskumar · 2024-11-19T19:38:10Z

!ci-build-me

co-authoring the PR

deepthiskumar · 2024-11-19T20:17:24Z

!ci-build-me

…-merkle-queries

mrmr1993 · 2024-11-20T11:34:01Z

src/lib/sync_handler/sync_handler.ml

+   fun ~frontier hash query ~context ~trust_system ->
+    let (module Context) = context in


Nit: ~context:(module Context)

mrmr1993 · 2024-11-20T11:39:37Z

src/lib/syncable_ledger/test/test.ml

+module Context_subtree_depth32 = Make_context (struct
+  let sync_ledger_max_subtree_depth = 3
+
+  let sync_ledger_default_subtree_depth = 2
+end)
+
+module Context_subtree_depth81 = Make_context (struct
+  let sync_ledger_max_subtree_depth = 8
+
+  let sync_ledger_default_subtree_depth = 1
+end)
+
+module Context_subtree_depth82 = Make_context (struct
+  let sync_ledger_max_subtree_depth = 8
+
+  let sync_ledger_default_subtree_depth = 2
+end)
+
+module Context_subtree_depth86 = Make_context (struct
+  let sync_ledger_max_subtree_depth = 8
+
+  let sync_ledger_default_subtree_depth = 6
+end)
+
+module Context_subtree_depth88 = Make_context (struct
+  let sync_ledger_max_subtree_depth = 8
+
+  let sync_ledger_default_subtree_depth = 8
+end)


It seems like there should also be a test for sync_ledger_max_subtree_depth < sync_ledger_default_subtree_depth.

mrmr1993 · 2024-11-20T11:44:24Z

src/lib/syncable_ledger/syncable_ledger.ml

@@ -44,15 +84,33 @@ module Answer = struct

      let to_latest acct_to_latest = function
        | Child_hashes_are (h1, h2) ->
-            Child_hashes_are (h1, h2)
+            V2.Child_hashes_are (List.to_array [ h1; h2 ])


Suggested change

V2.Child_hashes_are (List.to_array [ h1; h2 ])

V2.Child_hashes_are [| h1; h2 |]

mrmr1993 · 2024-11-20T11:47:48Z

src/lib/syncable_ledger/syncable_ledger.ml

+      let from_v2 : ('a, 'b) V2.t -> ('a, 'b) t = function
+        | Child_hashes_are h ->
+            if Array.length h = 2 then Child_hashes_are (h.(0), h.(1))
+            else failwith "can't downgrade wide query"


This really should return a Result IMO.

mrmr1993 · 2024-11-20T12:30:07Z

src/lib/syncable_ledger/syncable_ledger.ml

+        in
+        let left = intermediate_range ledger_depth left (i - 1) in
+        let right = intermediate_range ledger_depth right (i - 1) in
+        left @ right


Doing this list concatenation results in n^2 allocations. We also immediately convert it to an array immediately later. I'd recommend using an Array.init and appending the index, something like this.

mrmr1993 · 2024-11-20T12:32:11Z

src/lib/syncable_ledger/syncable_ledger.ml

+            match subtree_depth with
+            | n
+              when n >= 1
+                   && n <= Context.compile_config.sync_ledger_max_subtree_depth


This isn't a protocol violation. We should just take the min of the configured max and n.

mrmr1993 · 2024-11-20T12:33:58Z

src/lib/syncable_ledger/syncable_ledger.ml

+    let less_than_max =
+      len <= Int.pow 2 Context.compile_config.sync_ledger_max_subtree_depth


We don't care if it's less than our max. The only relevant maximum is the sending node's maximum.

I was thinking the receiving node wouldn't want to process some arbitrarily long list of hashes. Otherwise what is the need for specifying depth in the query if receiver accepts any depth?

The other test above checks that it's less than what we requested. The max is just what we're locally happy to respond with, unrelated to the responses we receive.

Oh wait, I added the requested depth check and this comment is for max. Sorry, you're right, we don't need the max

mrmr1993 · 2024-11-20T12:41:51Z

src/lib/syncable_ledger/syncable_ledger.ml

+    let (module Context) = t.context in
+    let open Context in


NB: you can do let open (val t.context).

…-merkle-queries

deepthiskumar · 2024-11-22T21:45:44Z

!ci-build-me

deepthiskumar · 2024-11-22T21:45:52Z

!ci-nightly-me

coveralls · 2024-11-23T03:08:48Z

coverage: 62.114%. first build
when pulling 36801fc on fabrizio-m/wide-merkle-queries
into c701a31 on compatible.

mrmr1993 · 2024-11-23T11:18:55Z

src/lib/mina_networking/rpcs.ml

+          return (Or_error.error_string "No Frontier")
    in
    let result =
-      Result.of_option answer
-        ~error:
-          (Error.createf
-             !"Refusing to answer sync ledger query for ledger_hash: \
-               %{sexp:Ledger_hash.t}"
-             ledger_hash )
+      Result.map_error answer ~f:(fun e ->
+          Error.createf
+            !"Refusing to answer sync ledger query for ledger_hash: \
+              %{sexp:Ledger_hash.t}. Error: %s"
+            ledger_hash (Error.to_string_hum e) )


This is confusing to me: why are we providing an error message No frontier in answer to just immediately replace it with map_error?

Relatedly, we then match on result below, so we will record Trust_system.Actions.Requested_unknown_item even if it's our fault for not having a frontier.

This behaviour is unchanged from before, so I won't block on it.

Yeah, I'm only propagating the actual error to test error cases.

we will record Trust_system.Actions.Requested_unknown_item even if it's our fault for not having a frontier.

This seems to be the behaviour for other messages too. Opened a ticket #16383

mrmr1993 · 2024-11-23T11:21:18Z

src/lib/syncable_ledger/test/test.ml

+  let sync_ledger_max_subtree_depth = 8
+
+  let sync_ledger_default_subtree_depth = 0


This test is a strange one: shouldn't we disallow default_subtree_depth = 0? If we request depth 0, I would understand that as meaning that we expect to get back [|hash|] when we request the depth-0 children of hash.

Yes, it is disallowed. It was an easy way to test the case. I could add a top level assertion for this in node_config and runtime config so that nodes aren't configured with 0 sub tree depth

mrmr1993 · 2024-11-23T11:23:20Z

src/lib/syncable_ledger/test/test.ml

+           match check_answer query answ_or_error with
+           | `Answer answ ->
+               let%bind () =
+                 if match query with What_contents _ -> true | _ -> false then


Nit: could use the match here directly.

mrmr1993 · 2024-11-23T11:25:14Z

src/lib/syncable_ledger/syncable_ledger.ml

+  (* TODO: parameterize *)
+  let subtree_depth : index = 4
+
+  (* Provides addresses at an specific depth from this address *)


Per this comment, nit:

Suggested change

(* Provides addresses at an specific depth from this address *)

(* Provides addresses at a specific depth from this address *)

mrmr1993 · 2024-11-23T11:31:06Z

src/lib/syncable_ledger/syncable_ledger.ml

+    let open (val t.context) in
+    let len = Array.length nodes in
+    let is_power = Int.is_pow2 len in
+    let is_more_than_two = len >= 2 in


Nit: is_more_than_two is misleading, 2 is not more than 2, but gives is_more_than_two = true.

mrmr1993 · 2024-11-23T11:31:30Z

src/lib/syncable_ledger/syncable_ledger.ml

+    let is_power = Int.is_pow2 len in
+    let is_more_than_two = len >= 2 in
+    let subtree_depth = Int.ceil_log2 len in
+    let less_than_requested = subtree_depth <= requested_depth in


Nit: similarly less_than_requested is not quite accurate.

mrmr1993 reviewed Sep 19, 2024

View reviewed changes

svv232 reviewed Sep 20, 2024

View reviewed changes

mrmr1993 reviewed Sep 23, 2024

View reviewed changes

fabrizio-m marked this pull request as ready for review September 23, 2024 16:14

fabrizio-m requested a review from a team as a code owner September 23, 2024 16:14

svv232 reviewed Sep 23, 2024

View reviewed changes

mrmr1993 requested changes Sep 24, 2024

View reviewed changes

fabrizio-m added 13 commits September 26, 2024 11:49

wide merkle query

d96eb19

missing comment

cc3d2bf

proper depth on range

c29b3f1

domain separation

f446563

handle answer

38fb17b

versioning

f5937a0

back to previous query

4f84b95

remove to_latest

ed16dab

unify in single query

77ab99b

check leaves are power of 2

d633026

removed unused functions

c54ff14

extra checks for length

250c655

handle exception

1229ef8

deepthiskumar added 3 commits November 11, 2024 22:15

update sync ledger unit tests with variable subtree depths

d26d015

move subtree depth constants to genesis constants, Runtime_config.dae…

db10629

…mon; use block window duration from runtime config and not compile config; remove compile_config from impacted contexts

Merge remote-tracking branch 'origin/compatible' into fabrizio-m/wide…

a27d3cc

…-merkle-queries

mrmr1993 requested changes Nov 13, 2024

View reviewed changes

svv232 reviewed Nov 13, 2024

View reviewed changes

src/lib/mina_ledger/sync_ledger.ml Outdated Show resolved Hide resolved

move sync ledger constants back to compile time

c14616d

deepthiskumar force-pushed the fabrizio-m/wide-merkle-queries branch from 0d0f6dc to c14616d Compare November 19, 2024 19:28

Merge remote-tracking branch 'origin/compatible' into fabrizio-m/wide…

a32c8ab

…-merkle-queries

fix typo in comment

8848b69

Co-authored-by: Sai Vegasena <[email protected]>

update syncable ledger tests

4a15962

Merge remote-tracking branch 'origin/compatible' into fabrizio-m/wide…

95f4052

…-merkle-queries

mrmr1993 requested changes Nov 20, 2024

View reviewed changes

deepthiskumar and others added 6 commits November 20, 2024 18:54

return sync ledger answer or error, not option

8f500b8

tests for requested subtree depth > max subtree depth

12fb304

return error if downgrading; other review comments

305fba1

remove max depth check by receiver; add tests for edge cases

45c6517

Add Merkle_address.extend, use it for array init in ledger sync

7732404

Merge remote-tracking branch 'origin/compatible' into fabrizio-m/wide…

36801fc

…-merkle-queries

mrmr1993 approved these changes Nov 23, 2024

View reviewed changes

deepthiskumar mentioned this pull request Nov 25, 2024

Follow up on comments from #16093 #16384

Open

7 tasks

mrmr1993 merged commit 05213da into compatible Nov 25, 2024
51 checks passed

mrmr1993 deleted the fabrizio-m/wide-merkle-queries branch November 25, 2024 12:31

		\| Subtree of 'hash list
		(** The subtree rooted on the requested address has these leaves *)

	(* Provides addresses at an specific depth from this address *)
	(* Provides addresses at a specific depth from this address *)

		(* TODO: give some error as this shouldn't really happen *)
		[]

		fun ~frontier hash query ~context ~trust_system ->
		let (module Context) = context in

	V2.Child_hashes_are (List.to_array [ h1; h2 ])
	V2.Child_hashes_are [\| h1; h2 \|]

		let less_than_max =
		len <= Int.pow 2 Context.compile_config.sync_ledger_max_subtree_depth

		let sync_ledger_max_subtree_depth = 8

		let sync_ledger_default_subtree_depth = 0

Wide merkle query #16093

Wide merkle query #16093

Conversation

fabrizio-m commented Sep 19, 2024 • edited by deepthiskumar Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fabrizio-m commented Sep 23, 2024

fabrizio-m commented Sep 23, 2024

fabrizio-m commented Sep 23, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

svv232 Sep 23, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mrmr1993 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fabrizio-m commented Sep 24, 2024

fabrizio-m commented Sep 24, 2024

mrmr1993 commented Sep 25, 2024

fabrizio-m commented Sep 25, 2024

fabrizio-m commented Sep 25, 2024

fabrizio-m commented Sep 25, 2024

fabrizio-m commented Sep 26, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

deepthiskumar Nov 19, 2024 • edited Loading

Choose a reason for hiding this comment

deepthiskumar commented Nov 19, 2024

deepthiskumar commented Nov 19, 2024

deepthiskumar commented Nov 19, 2024

deepthiskumar commented Nov 19, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mrmr1993 Nov 20, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

deepthiskumar commented Nov 22, 2024

deepthiskumar commented Nov 22, 2024

coveralls commented Nov 23, 2024

fabrizio-m commented Sep 19, 2024 •

edited by deepthiskumar

Loading

svv232 Sep 23, 2024 •

edited

Loading

deepthiskumar Nov 19, 2024 •

edited

Loading

mrmr1993 Nov 20, 2024 •

edited

Loading