Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enforce no_std for all? protocol/v2 crates #1230

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

Georges760
Copy link

@Georges760 Georges760 commented Oct 19, 2024

Most of protocol/v2 crates are almost no_std, as listed here.

Having a lib crate that can be no_std or std by the selection of a feature :

#![cfg_attr(not(feature = "std"), no_std)]

or the current implementation :

#![cfg_attr(feature = "no_std", no_std)]

is only usefull if :

  • the crate could use some std related dependency (for instance std::vec::Vec)
  • but we want the crate's user to be able to not have a dependancy on std so we disable the functionalities using std

In some case we can replace the std dep by its alloc equivalent if exists (alloc::vec::Vec) and we can keep the functionality in no_std environement.

Finally, a std binary can use some no_std dependencies, but an no_std binary cannot use any std deps. So making the low level crates all no_std will make them copamtible for every kind of binary (app).

We should use the std feature only if some functionality requires std and not have any no_std equivalent.

As far as I tried all protocol/v2/ crates (libs) can be 100% no_std a,d does not need the std (or no_std) feature to optionally add std dep.

This PR is targeting to get rid of std dep without loosing any functionality or efficiency.

Note: I will make 1 commit per crate to explain what and how I am doing it.
Note2: being no_alloc is another level of complexity for deep embedded system, which is not at all this PR concern.

Copy link
Contributor

github-actions bot commented Oct 19, 2024

🐰 Bencher Report

Branch1230/merge
Testbedsv2

⚠️ WARNING: The following Measures do not have a Threshold. Without a Threshold, no Alerts will ever be generated!

Click here to create a new Threshold
For more information, see the Threshold documentation.
To only post results if a Threshold exists, set the --ci-only-thresholds CLI flag.

Click to view all benchmark results
BenchmarkEstimated Cyclesestimated cyclesInstructionsinstructionsL1 AccessesaccessesL2 AccessesaccessesRAM Accessesaccesses
client_sv2_handle_message_common📈 view plot
⚠️ NO THRESHOLD
2,073.00📈 view plot
⚠️ NO THRESHOLD
473.00📈 view plot
⚠️ NO THRESHOLD
733.00📈 view plot
⚠️ NO THRESHOLD
9.00📈 view plot
⚠️ NO THRESHOLD
37.00
client_sv2_handle_message_mining📈 view plot
⚠️ NO THRESHOLD
8,178.00📈 view plot
⚠️ NO THRESHOLD
2,137.00📈 view plot
⚠️ NO THRESHOLD
3,158.00📈 view plot
⚠️ NO THRESHOLD
38.00📈 view plot
⚠️ NO THRESHOLD
138.00
client_sv2_mining_message_submit_standard📈 view plot
⚠️ NO THRESHOLD
6,263.00📈 view plot
⚠️ NO THRESHOLD
1,750.00📈 view plot
⚠️ NO THRESHOLD
2,548.00📈 view plot
⚠️ NO THRESHOLD
22.00📈 view plot
⚠️ NO THRESHOLD
103.00
client_sv2_mining_message_submit_standard_serialize📈 view plot
⚠️ NO THRESHOLD
14,694.00📈 view plot
⚠️ NO THRESHOLD
4,694.00📈 view plot
⚠️ NO THRESHOLD
6,754.00📈 view plot
⚠️ NO THRESHOLD
48.00📈 view plot
⚠️ NO THRESHOLD
220.00
client_sv2_mining_message_submit_standard_serialize_deserialize📈 view plot
⚠️ NO THRESHOLD
27,746.00📈 view plot
⚠️ NO THRESHOLD
10,645.00📈 view plot
⚠️ NO THRESHOLD
15,506.00📈 view plot
⚠️ NO THRESHOLD
89.00📈 view plot
⚠️ NO THRESHOLD
337.00
client_sv2_open_channel📈 view plot
⚠️ NO THRESHOLD
4,413.00📈 view plot
⚠️ NO THRESHOLD
1,461.00📈 view plot
⚠️ NO THRESHOLD
2,158.00📈 view plot
⚠️ NO THRESHOLD
10.00📈 view plot
⚠️ NO THRESHOLD
63.00
client_sv2_open_channel_serialize📈 view plot
⚠️ NO THRESHOLD
14,024.00📈 view plot
⚠️ NO THRESHOLD
5,064.00📈 view plot
⚠️ NO THRESHOLD
7,324.00📈 view plot
⚠️ NO THRESHOLD
38.00📈 view plot
⚠️ NO THRESHOLD
186.00
client_sv2_open_channel_serialize_deserialize📈 view plot
⚠️ NO THRESHOLD
22,832.00📈 view plot
⚠️ NO THRESHOLD
8,040.00📈 view plot
⚠️ NO THRESHOLD
11,687.00📈 view plot
⚠️ NO THRESHOLD
87.00📈 view plot
⚠️ NO THRESHOLD
306.00
client_sv2_setup_connection📈 view plot
⚠️ NO THRESHOLD
4,659.00📈 view plot
⚠️ NO THRESHOLD
1,502.00📈 view plot
⚠️ NO THRESHOLD
2,274.00📈 view plot
⚠️ NO THRESHOLD
15.00📈 view plot
⚠️ NO THRESHOLD
66.00
client_sv2_setup_connection_serialize📈 view plot
⚠️ NO THRESHOLD
16,134.00📈 view plot
⚠️ NO THRESHOLD
5,963.00📈 view plot
⚠️ NO THRESHOLD
8,659.00📈 view plot
⚠️ NO THRESHOLD
46.00📈 view plot
⚠️ NO THRESHOLD
207.00
client_sv2_setup_connection_serialize_deserialize📈 view plot
⚠️ NO THRESHOLD
35,670.00📈 view plot
⚠️ NO THRESHOLD
14,888.00📈 view plot
⚠️ NO THRESHOLD
21,880.00📈 view plot
⚠️ NO THRESHOLD
98.00📈 view plot
⚠️ NO THRESHOLD
380.00
🐰 View full continuous benchmarking report in Bencher

Copy link
Contributor

github-actions bot commented Oct 19, 2024

🐰 Bencher Report

Branch1230/merge
Testbedsv2

⚠️ WARNING: The following Measure does not have a Threshold. Without a Threshold, no Alerts will ever be generated!

Click here to create a new Threshold
For more information, see the Threshold documentation.
To only post results if a Threshold exists, set the --ci-only-thresholds CLI flag.

Click to view all benchmark results
BenchmarkLatencynanoseconds (ns)
client_sv2_handle_message_common📈 view plot
⚠️ NO THRESHOLD
44.06
client_sv2_handle_message_mining📈 view plot
⚠️ NO THRESHOLD
75.36
client_sv2_mining_message_submit_standard📈 view plot
⚠️ NO THRESHOLD
14.67
client_sv2_mining_message_submit_standard_serialize📈 view plot
⚠️ NO THRESHOLD
256.91
client_sv2_mining_message_submit_standard_serialize_deserialize📈 view plot
⚠️ NO THRESHOLD
612.34
client_sv2_open_channel📈 view plot
⚠️ NO THRESHOLD
148.15
client_sv2_open_channel_serialize📈 view plot
⚠️ NO THRESHOLD
277.25
client_sv2_open_channel_serialize_deserialize📈 view plot
⚠️ NO THRESHOLD
380.26
client_sv2_setup_connection📈 view plot
⚠️ NO THRESHOLD
161.96
client_sv2_setup_connection_serialize📈 view plot
⚠️ NO THRESHOLD
455.68
client_sv2_setup_connection_serialize_deserialize📈 view plot
⚠️ NO THRESHOLD
957.41
🐰 View full continuous benchmarking report in Bencher

Copy link
Contributor

github-actions bot commented Oct 19, 2024

🐰 Bencher Report

Branch1230/merge
Testbedsv1

⚠️ WARNING: The following Measure does not have a Threshold. Without a Threshold, no Alerts will ever be generated!

Click here to create a new Threshold
For more information, see the Threshold documentation.
To only post results if a Threshold exists, set the --ci-only-thresholds CLI flag.

Click to view all benchmark results
BenchmarkLatencynanoseconds (ns)
client-submit-serialize📈 view plot
⚠️ NO THRESHOLD
6,748.30
client-submit-serialize-deserialize📈 view plot
⚠️ NO THRESHOLD
7,707.10
client-submit-serialize-deserialize-handle/client-submit-serialize-deserialize-handle📈 view plot
⚠️ NO THRESHOLD
8,377.50
client-sv1-authorize-serialize-deserialize-handle/client-sv1-authorize-serialize-deserialize-handle📈 view plot
⚠️ NO THRESHOLD
943.45
client-sv1-authorize-serialize-deserialize/client-sv1-authorize-serialize-deserialize📈 view plot
⚠️ NO THRESHOLD
675.37
client-sv1-authorize-serialize/client-sv1-authorize-serialize📈 view plot
⚠️ NO THRESHOLD
262.34
client-sv1-get-authorize/client-sv1-get-authorize📈 view plot
⚠️ NO THRESHOLD
156.57
client-sv1-get-submit📈 view plot
⚠️ NO THRESHOLD
6,451.40
client-sv1-get-subscribe/client-sv1-get-subscribe📈 view plot
⚠️ NO THRESHOLD
289.67
client-sv1-subscribe-serialize-deserialize-handle/client-sv1-subscribe-serialize-deserialize-handle📈 view plot
⚠️ NO THRESHOLD
812.52
client-sv1-subscribe-serialize-deserialize/client-sv1-subscribe-serialize-deserialize📈 view plot
⚠️ NO THRESHOLD
607.60
client-sv1-subscribe-serialize/client-sv1-subscribe-serialize📈 view plot
⚠️ NO THRESHOLD
222.88
🐰 View full continuous benchmarking report in Bencher

Copy link
Contributor

github-actions bot commented Oct 19, 2024

🐰 Bencher Report

Branch1230/merge
Testbedsv1

⚠️ WARNING: The following Measures do not have a Threshold. Without a Threshold, no Alerts will ever be generated!

Click here to create a new Threshold
For more information, see the Threshold documentation.
To only post results if a Threshold exists, set the --ci-only-thresholds CLI flag.

Click to view all benchmark results
BenchmarkEstimated Cyclesestimated cyclesInstructionsinstructionsL1 AccessesaccessesL2 AccessesaccessesRAM Accessesaccesses
get_authorize📈 view plot
⚠️ NO THRESHOLD
8,304.00📈 view plot
⚠️ NO THRESHOLD
3,663.00📈 view plot
⚠️ NO THRESHOLD
5,114.00📈 view plot
⚠️ NO THRESHOLD
8.00📈 view plot
⚠️ NO THRESHOLD
90.00
get_submit📈 view plot
⚠️ NO THRESHOLD
95,226.00📈 view plot
⚠️ NO THRESHOLD
59,263.00📈 view plot
⚠️ NO THRESHOLD
85,076.00📈 view plot
⚠️ NO THRESHOLD
49.00📈 view plot
⚠️ NO THRESHOLD
283.00
get_subscribe📈 view plot
⚠️ NO THRESHOLD
7,871.00📈 view plot
⚠️ NO THRESHOLD
2,758.00📈 view plot
⚠️ NO THRESHOLD
3,831.00📈 view plot
⚠️ NO THRESHOLD
17.00📈 view plot
⚠️ NO THRESHOLD
113.00
serialize_authorize📈 view plot
⚠️ NO THRESHOLD
12,041.00📈 view plot
⚠️ NO THRESHOLD
5,240.00📈 view plot
⚠️ NO THRESHOLD
7,281.00📈 view plot
⚠️ NO THRESHOLD
14.00📈 view plot
⚠️ NO THRESHOLD
134.00
serialize_deserialize_authorize📈 view plot
⚠️ NO THRESHOLD
24,452.00📈 view plot
⚠️ NO THRESHOLD
9,786.00📈 view plot
⚠️ NO THRESHOLD
13,787.00📈 view plot
⚠️ NO THRESHOLD
40.00📈 view plot
⚠️ NO THRESHOLD
299.00
serialize_deserialize_handle_authorize📈 view plot
⚠️ NO THRESHOLD
30,171.00📈 view plot
⚠️ NO THRESHOLD
11,989.00📈 view plot
⚠️ NO THRESHOLD
16,951.00📈 view plot
⚠️ NO THRESHOLD
61.00📈 view plot
⚠️ NO THRESHOLD
369.00
serialize_deserialize_handle_submit📈 view plot
⚠️ NO THRESHOLD
126,220.00📈 view plot
⚠️ NO THRESHOLD
73,117.00📈 view plot
⚠️ NO THRESHOLD
104,760.00📈 view plot
⚠️ NO THRESHOLD
120.00📈 view plot
⚠️ NO THRESHOLD
596.00
serialize_deserialize_handle_subscribe📈 view plot
⚠️ NO THRESHOLD
27,761.00📈 view plot
⚠️ NO THRESHOLD
9,577.00📈 view plot
⚠️ NO THRESHOLD
13,521.00📈 view plot
⚠️ NO THRESHOLD
69.00📈 view plot
⚠️ NO THRESHOLD
397.00
serialize_deserialize_submit📈 view plot
⚠️ NO THRESHOLD
114,825.00📈 view plot
⚠️ NO THRESHOLD
67,894.00📈 view plot
⚠️ NO THRESHOLD
97,370.00📈 view plot
⚠️ NO THRESHOLD
68.00📈 view plot
⚠️ NO THRESHOLD
489.00
serialize_deserialize_subscribe📈 view plot
⚠️ NO THRESHOLD
23,052.00📈 view plot
⚠️ NO THRESHOLD
8,129.00📈 view plot
⚠️ NO THRESHOLD
11,427.00📈 view plot
⚠️ NO THRESHOLD
43.00📈 view plot
⚠️ NO THRESHOLD
326.00
serialize_submit📈 view plot
⚠️ NO THRESHOLD
99,505.00📈 view plot
⚠️ NO THRESHOLD
61,325.00📈 view plot
⚠️ NO THRESHOLD
87,940.00📈 view plot
⚠️ NO THRESHOLD
52.00📈 view plot
⚠️ NO THRESHOLD
323.00
serialize_subscribe📈 view plot
⚠️ NO THRESHOLD
11,176.00📈 view plot
⚠️ NO THRESHOLD
4,111.00📈 view plot
⚠️ NO THRESHOLD
5,696.00📈 view plot
⚠️ NO THRESHOLD
18.00📈 view plot
⚠️ NO THRESHOLD
154.00
🐰 View full continuous benchmarking report in Bencher

Copy link

codecov bot commented Oct 19, 2024

Codecov Report

Attention: Patch coverage is 50.00000% with 1 line in your changes missing coverage. Please review.

Project coverage is 25.13%. Comparing base (73f51db) to head (461d753).
Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
utils/buffer/src/buffer_pool/mod.rs 50.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1230      +/-   ##
==========================================
+ Coverage   19.36%   25.13%   +5.76%     
==========================================
  Files         164       20     -144     
  Lines       10811     1134    -9677     
==========================================
- Hits         2094      285    -1809     
+ Misses       8717      849    -7868     
Flag Coverage Δ
binary_codec_sv2-coverage ?
binary_serde_sv2-coverage ?
binary_sv2-coverage ?
bip32_derivation-coverage ?
buffer_sv2-coverage ?
codec_sv2-coverage ?
common_messages_sv2-coverage ?
const_sv2-coverage ?
error_handling-coverage ?
framing_sv2-coverage ?
jd_client-coverage ?
jd_server-coverage ?
job_declaration_sv2-coverage ?
key-utils-coverage ?
mining-coverage ?
mining_device-coverage ?
mining_proxy_sv2-coverage ?
noise_sv2-coverage ?
pool_sv2-coverage ?
protocols ?
roles ?
roles_logic_sv2-coverage ?
sv1-mining-device-coverage ?
sv2_ffi-coverage ?
template_distribution_sv2-coverage ?
translator_sv2-coverage ?
utils 25.13% <50.00%> (ø)
v1-coverage ?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

protocols/DEPENDANCIES.md Outdated Show resolved Hide resolved
@Georges760 Georges760 changed the title std logic enforce no_std for all protocol/v2 crates Oct 28, 2024
Copy link
Contributor

@jbesraa jbesraa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While I think the general idea is very welcome, I think addressing #1200 first would make more sense as it can potentially change how we approach this. I think in #1200 we should aim to provide a logger for std and no std env and then impl Debug manually, which would allow us to remove the debug feature IIUC whats going here

@Georges760
Copy link
Author

Current binary_codec_sv2 :

  • don't compile without no_std feature (this doesn't mean it is no_std as there is no enforcement => #![cfg_attr(feature = "no_std", no_std)])
  • has no_std feature by default (default = ["no_std"])
  • nowhere (not even in the CI), its default features are disable (--no-default-features)

So it's pseudo std support os broken and unused. We can :

  • remove/clean it, and keep a real no_std crate only
  • debug it, and have a no_std crate by default, with a std feature with the debugged std functionalities

Maybe this can be discussed in #1132 ?

@Georges760
Copy link
Author

While I think the general idea is very welcome, I think addressing #1200 first would make more sense as it can potentially change how we approach this. I think in #1200 we should aim to provide a logger for std and no std env and then impl Debug manually, which would allow us to remove the debug feature IIUC whats going here

tracing has a std feature enabled by default, this means it actually add std dep to the crate that use it without disabling default features : sv1_api, binary_sv2, codec_sv2 and roles_logic_sv2 (the 4 crates listed in #1200). 2 of this 4 crates are targeted by this PR (binary_sv2 and codec_sv2) and in order for them to become no_std, I will have to no-default-features tracing, and enable std in tracing using a local feature too. This can be done before or after #1200 without problem.

If I do it before #1200, when #1200 will remove tracing with the custom logger, the std feature will be available (and can be removed if not used after all).
If #1200 should be done before, then we can limit this PR to crates not depending on binary_sv2 and codec_sv2, and wait to see what the new logger will propose for no_std support.

@Georges760 Georges760 changed the title enforce no_std for all protocol/v2 crates enforce no_std for all? protocol/v2 crates Oct 28, 2024
@jbesraa
Copy link
Contributor

jbesraa commented Oct 28, 2024

While I think the general idea is very welcome, I think addressing #1200 first would make more sense as it can potentially change how we approach this. I think in #1200 we should aim to provide a logger for std and no std env and then impl Debug manually, which would allow us to remove the debug feature IIUC whats going here

tracing has a std feature enabled by default, this means it actually add std dep to the crate that use it without disabling default features : sv1_api, binary_sv2, codec_sv2 and roles_logic_sv2 (the 4 crates listed in #1200). 2 of this 4 crates are targeted by this PR (binary_sv2 and codec_sv2) and in order for them to become no_std, I will have to no-default-features tracing, and enable std in tracing using a local feature too. This can be done before or after #1200 without problem.

If I do it before #1200, when #1200 will remove tracing with the custom logger, the std feature will be available (and can be removed if not used after all). If #1200 should be done before, then we can limit this PR to crates not depending on binary_sv2 and codec_sv2, and wait to see what the new logger will propose for no_std support.

Yea I think #1200 shouldn't necessarily be about removing tracing crate, but generalizing how we use the logger
and mainly enabling a logger in a non-std env. Making everything non-std by default would make it a bit tricky to debug the crates generally.
With that said, If #1200 will require too much time and would block this for too long, I am happy for this to get in anyway.

@Georges760 Georges760 force-pushed the std_logic branch 3 times, most recently from dfb23fe to 8ef9d5e Compare October 28, 2024 14:10
@Georges760
Copy link
Author

Actual binary_sv2 without default feature (core) and with with_serde feature, test is broken.

This PR is not fixing this.

CI does not see it because does not test it.

@Georges760
Copy link
Author

Actual framing_sv2 with with_serde feature, test is broken.

This PR is not fixing this.

CI does not see it because does not test it.

@plebhash
Copy link
Collaborator

@Georges760 please note that we're aiming to deprecate with_serde feature soon.

I cannot give you a precise timeline but Q1 2025 would be my rough guess.

I wonder if that would make this process simpler?

@Georges760
Copy link
Author

@Georges760 please note that we're aiming to deprecate with_serde feature soon.

I cannot give you a precise timeline but Q1 2025 would be my rough guess.

I wonder if that would make this process simpler?

It make sense, as it is already broken in binary_sv2 and framing_sv2 as stated above.

So I understand the official/only codec way will be the "core" option ? Good

@Georges760
Copy link
Author

noise_sv2 has its own PR #1238.
codec_sv2 is dependant on both noise_sv2 and all the crates of this PR.

So I stop here for this PR and open for reviews.
I think it is in good shape to fullfil the OP goal.

@Georges760 Georges760 marked this pull request as ready for review October 29, 2024 11:08
protocols/v2/binary-sv2/binary-sv2/src/lib.rs Outdated Show resolved Hide resolved
@@ -19,6 +19,9 @@
//! Seq0255 <-> SEQ0_255[T]
//! Seq064K <-> SEQ0_64K[T]
//! ```

#![cfg_attr(feature = "no_std", no_std)]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can now use #![no_std], and I'm confident that we are not using any std-compliant methods in binary_codec_sv2 in any of the top-level crates that depend on it. Since almost all of our crates are no_std-compliant, makes sense to have no-serde-sv2 to no-std only. cc: @rrybarczyk

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I asked about this in this comment.

Current binary_codec_sv2 has a broken std support, do we want to remove it or fix it ?

Look like you propose to remove it. It is also my preference.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also think we should enforce no_std on the protocols crates. So let's remove.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR looks good to me and is ready to go. We can enforce the no-std restriction during our protocol refactoring effort, once we have more concrete reasoning for removing std-related imports. However, I’m fine with either approach.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree. Let's confirm in the dev meet before we approve.

Georges Palauqui added 8 commits October 29, 2024 15:26
- only feature `debug` will add `std` dep which is not used by any other crates
- just had to replace `std::hint` by its `core` equivalent
- in `debug` removed a `mut` to clear a warning
- std::vec::Vec -> alloc::vec::Vec
- std::mem -> core::mem
- std::slice -> core::slice
- std::convert::{TryFrom, TryInto} -> core::convert::{TryFrom, TryInto}
- std::format -> alloc::format
- std::vec::Vec -> alloc::vec::Vec
- std::string::{String, ToString} -> alloc::string::{String, ToString}
no need any feature for `std` stuff, there is none
- std::vec::Vec -> alloc::vec::Vec
@plebhash
Copy link
Collaborator

plebhash commented Nov 5, 2024

completely removing std from protocols crates will imply making adaptations to the fuzz tests

@rrybarczyk rrybarczyk added the protocols Lowest level protocol logic label Nov 11, 2024
@plebhash
Copy link
Collaborator

plebhash commented Nov 11, 2024

As far as I tried all protocol/v2/ crates (libs) can be 100% no_std a,d does not need the std (or no_std) feature to optionally add std dep.

if I understand correctly, we should expect this PR to leave all protocols crates (with exception of noise_sv2 + codec_sv2, which will be covered in other PRs) without ANY std/no_std feature flag, since they will be no_std by default.

but I'm confused:

why are we still leaving some crates with the no_std feature flag? I inspected this branch and found the no_std feature flag on the following crates:

  • v2/codec-sv2/Cargo.toml (expected, since this PR doesn't aim to touch this crate IIUC)
  • v2/subprotocols/job-declaration/Cargo.toml
  • v2/subprotocols/mining/Cargo.toml
  • v2/binary-sv2/no-serde-sv2/codec/Cargo.toml
  • v2/subprotocols/template-distribution/Cargo.toml
  • v2/subprotocols/common-messages/Cargo.toml

shouldn't no_std feature flag be completely removed from these crates?
shouldn't they KISS and only have a #![no_std] on the top of the module, instead of #![cfg_attr(feature = "no_std", no_std)]? e.g.: 422aee3

@plebhash
Copy link
Collaborator

plebhash commented Nov 12, 2024

following this criteria, we should bump the PATCH version of the crates that are being changed on this PR

@Georges760
Copy link
Author

As far as I tried all protocol/v2/ crates (libs) can be 100% no_std a,d does not need the std (or no_std) feature to optionally add std dep.

if I understand correctly, we should expect this PR to leave all protocols crates (with exception of noise_sv2 + codec_sv2, which will be covered in other PRs) without ANY std/no_std feature flag, since they will be no_std by default.

That is the end goal yes

but I'm confused:

why are we still leaving some crates with the no_std feature flag? I inspected this branch and found the no_std feature flag on the following crates:

  • v2/codec-sv2/Cargo.toml (expected, since this PR doesn't aim to touch this crate IIUC)
  • v2/subprotocols/job-declaration/Cargo.toml
  • v2/subprotocols/mining/Cargo.toml

I didn't consider subprotocols/* crates either in this PR

  • v2/binary-sv2/no-serde-sv2/codec/Cargo.toml

details here above
TL;DR: kept no_std feature for an existing yet broken implementation (don't want to remove any piece of code, even if doesn't compile, but yes I think we should also clean what is under the #[cfg(not(feature = "no_std"))] in this crate, maybe part of #1132 )

  • v2/subprotocols/template-distribution/Cargo.toml
  • v2/subprotocols/common-messages/Cargo.toml

shouldn't no_std feature flag be completely removed from these crates? shouldn't they KISS and only have a #![no_std] on the top of the module, instead of #![cfg_attr(feature = "no_std", no_std)]? e.g.: 422aee3

YES totally, but this PR being multi crate already, try to limit its action to not refactor any code, just remove std dependencies where possible without functional impact.

@Georges760
Copy link
Author

following this criteria, we should bump the PATCH version of the crates that are being changed on this PR

exact, but usually the version bump is done in a separate PR/commit to merge multiple improvement in a single version increment. Should I do it in this PR anyway ?

Copy link
Contributor

@Shourya742 Shourya742 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK

@plebhash
Copy link
Collaborator

following this criteria, we should bump the PATCH version of the crates that are being changed on this PR

exact, but usually the version bump is done in a separate PR/commit to merge multiple improvement in a single version increment. Should I do it in this PR anyway ?

you're right, we can leave it for a new PR

@rrybarczyk rrybarczyk self-requested a review November 12, 2024 17:30
@plebhash
Copy link
Collaborator

@Georges760 could you please add a comment to the crates README.md mentioning that they are no_std by default?

some crates are missing a README.md (WIP under #845)

for the crates that are missing a README.md, feel free to create the file and only add this sentence

later, we're going to finish the README.md file with the rest of the info

Copy link
Contributor

🐰 Bencher Report

Branchstd_logic
Testbedsv1
Click to view all benchmark results
BenchmarkLatencyBenchmark Result
nanoseconds (ns)
(Result Δ%)
Upper Boundary
nanoseconds (ns)
(Limit %)
client-submit-serialize📈 view plot
🚷 view threshold
6,753.30
(-1.44%)
9,314.18
(72.51%)
client-submit-serialize-deserialize📈 view plot
🚷 view threshold
7,671.70
(-1.17%)
10,337.28
(74.21%)
client-submit-serialize-deserialize-handle/client-submit-serialize-deserialize-handle📈 view plot
🚷 view threshold
8,195.80
(-1.90%)
11,221.66
(73.04%)
client-sv1-authorize-serialize-deserialize-handle/client-sv1-authorize-serialize-deserialize-handle📈 view plot
🚷 view threshold
898.90
(-0.44%)
1,234.36
(72.82%)
client-sv1-authorize-serialize-deserialize/client-sv1-authorize-serialize-deserialize📈 view plot
🚷 view threshold
702.36
(-0.13%)
1,011.05
(69.47%)
client-sv1-authorize-serialize/client-sv1-authorize-serialize📈 view plot
🚷 view threshold
247.04
(-0.48%)
275.73
(89.59%)
client-sv1-get-authorize/client-sv1-get-authorize📈 view plot
🚷 view threshold
159.33
(+1.19%)
166.80
(95.52%)
client-sv1-get-submit📈 view plot
🚷 view threshold
6,554.50
(-0.85%)
9,121.75
(71.86%)
client-sv1-get-subscribe/client-sv1-get-subscribe📈 view plot
🚷 view threshold
278.44
(-0.07%)
335.87
(82.90%)
client-sv1-subscribe-serialize-deserialize-handle/client-sv1-subscribe-serialize-deserialize-handle📈 view plot
🚷 view threshold
742.61
(-2.97%)
993.77
(74.73%)
client-sv1-subscribe-serialize-deserialize/client-sv1-subscribe-serialize-deserialize📈 view plot
🚷 view threshold
610.13
(-2.00%)
847.81
(71.97%)
client-sv1-subscribe-serialize/client-sv1-subscribe-serialize📈 view plot
🚷 view threshold
202.84
(-1.32%)
231.84
(87.49%)
🐰 View full continuous benchmarking report in Bencher

Copy link
Contributor

🐰 Bencher Report

Branchstd_logic
Testbedsv2
🚨 7 ALERTS: Threshold Boundary Limits exceeded!
BenchmarkMeasure
Units
ViewBenchmark Result
(Result Δ%)
Lower Boundary
(Limit %)
Upper Boundary
(Limit %)
client_sv2_mining_message_submit_standard_serializeL2 Accesses
accesses
📈 plot
🚨 alert
🚷 threshold
54.00
(+23.78%)
53.93
(100.14%)
client_sv2_mining_message_submit_standard_serialize_deserializeEstimated Cycles
estimated cycles
📈 plot
🚨 alert
🚷 threshold
27,826.00
(+1.07%)
27,744.72
(100.29%)
client_sv2_mining_message_submit_standard_serialize_deserializeInstructions
instructions
📈 plot
🚨 alert
🚷 threshold
10,645.00
(+0.64%)
10,638.14
(100.06%)
client_sv2_mining_message_submit_standard_serialize_deserializeL1 Accesses
accesses
📈 plot
🚨 alert
🚷 threshold
15,501.00
(+0.72%)
15,484.71
(100.11%)
client_sv2_mining_message_submit_standard_serialize_deserializeL2 Accesses
accesses
📈 plot
🚨 alert
🚷 threshold
92.00
(+13.23%)
89.87
(102.37%)
client_sv2_open_channel_serialize_deserializeL2 Accesses
accesses
📈 plot
🚨 alert
🚷 threshold
90.00
(+21.39%)
85.23
(105.60%)
client_sv2_setup_connection_serialize_deserializeEstimated Cycles
estimated cycles
📈 plot
🚨 alert
🚷 threshold
35,750.00
(+0.55%)
35,745.38
(100.01%)
Click to view all benchmark results
BenchmarkEstimated CyclesBenchmark Result
estimated cycles
(Result Δ%)
Upper Boundary
estimated cycles
(Limit %)
InstructionsBenchmark Result
instructions
(Result Δ%)
Upper Boundary
instructions
(Limit %)
L1 AccessesBenchmark Result
accesses
(Result Δ%)
Upper Boundary
accesses
(Limit %)
L2 AccessesBenchmark Result
accesses
(Result Δ%)
Upper Boundary
accesses
(Limit %)
RAM AccessesBenchmark Result
accesses
(Result Δ%)
Upper Boundary
accesses
(Limit %)
client_sv2_handle_message_common📈 view plot
🚷 view threshold
2,163.00
(+3.37%)
2,205.43
(98.08%)
📈 view plot
🚷 view threshold
473.00
(-0.10%)
490.80
(96.37%)
📈 view plot
🚷 view threshold
733.00
(-0.41%)
760.98
(96.32%)
📈 view plot
🚷 view threshold
6.00
(+8.17%)
14.27
(42.05%)
📈 view plot
🚷 view threshold
40.00
(+5.37%)
41.13
(97.26%)
client_sv2_handle_message_mining📈 view plot
🚷 view threshold
8,208.00
(-0.01%)
8,298.83
(98.91%)
📈 view plot
🚷 view threshold
2,137.00
(-0.03%)
2,143.08
(99.72%)
📈 view plot
🚷 view threshold
3,158.00
(-0.09%)
3,170.64
(99.60%)
📈 view plot
🚷 view threshold
37.00
(+3.52%)
42.71
(86.63%)
📈 view plot
🚷 view threshold
139.00
(-0.08%)
141.15
(98.48%)
client_sv2_mining_message_submit_standard📈 view plot
🚷 view threshold
6,357.00
(+1.22%)
6,387.17
(99.53%)
📈 view plot
🚷 view threshold
1,750.00
(-0.06%)
1,769.03
(98.92%)
📈 view plot
🚷 view threshold
2,547.00
(-0.28%)
2,579.96
(98.72%)
📈 view plot
🚷 view threshold
20.00
(+17.76%)
24.81
(80.60%)
📈 view plot
🚷 view threshold
106.00
(+1.88%)
106.98
(99.09%)
client_sv2_mining_message_submit_standard_serialize📈 view plot
🚷 view threshold
14,820.00
(+0.55%)
14,996.55
(98.82%)
📈 view plot
🚷 view threshold
4,694.00
(-0.02%)
4,713.03
(99.60%)
📈 view plot
🚷 view threshold
6,745.00
(-0.20%)
6,785.54
(99.40%)
📈 view plot
🚨 view alert
🚷 view threshold
54.00
(+23.78%)
53.93
(100.14%)
📈 view plot
🚷 view threshold
223.00
(+0.54%)
228.54
(97.58%)
client_sv2_mining_message_submit_standard_serialize_deserialize📈 view plot
🚨 view alert
🚷 view threshold
27,826.00
(+1.07%)
27,744.72
(100.29%)
📈 view plot
🚨 view alert
🚷 view threshold
10,645.00
(+0.64%)
10,638.14
(100.06%)
📈 view plot
🚨 view alert
🚷 view threshold
15,501.00
(+0.72%)
15,484.71
(100.11%)
📈 view plot
🚨 view alert
🚷 view threshold
92.00
(+13.23%)
89.87
(102.37%)
📈 view plot
🚷 view threshold
339.00
(+1.11%)
342.17
(99.07%)
client_sv2_open_channel📈 view plot
🚷 view threshold
4,451.00
(+1.06%)
4,632.44
(96.08%)
📈 view plot
🚷 view threshold
1,461.00
(-0.03%)
1,478.80
(98.80%)
📈 view plot
🚷 view threshold
2,156.00
(-0.17%)
2,187.48
(98.56%)
📈 view plot
🚷 view threshold
11.00
(+20.65%)
17.96
(61.25%)
📈 view plot
🚷 view threshold
64.00
(+1.87%)
68.89
(92.90%)
client_sv2_open_channel_serialize📈 view plot
🚷 view threshold
14,066.00
(-0.06%)
14,491.35
(97.06%)
📈 view plot
🚷 view threshold
5,064.00
(-0.01%)
5,081.80
(99.65%)
📈 view plot
🚷 view threshold
7,321.00
(-0.06%)
7,355.03
(99.54%)
📈 view plot
🚷 view threshold
40.00
(+13.88%)
43.21
(92.57%)
📈 view plot
🚷 view threshold
187.00
(-0.43%)
199.79
(93.60%)
client_sv2_open_channel_serialize_deserialize📈 view plot
🚷 view threshold
22,912.00
(+1.08%)
22,916.56
(99.98%)
📈 view plot
🚷 view threshold
8,040.00
(+0.27%)
8,075.38
(99.56%)
📈 view plot
🚷 view threshold
11,682.00
(+0.15%)
11,756.45
(99.37%)
📈 view plot
🚨 view alert
🚷 view threshold
90.00
(+21.39%)
85.23
(105.60%)
📈 view plot
🚷 view threshold
308.00
(+1.38%)
312.64
(98.52%)
client_sv2_setup_connection📈 view plot
🚷 view threshold
4,723.00
(+0.80%)
4,777.66
(98.86%)
📈 view plot
🚷 view threshold
1,502.00
(-0.03%)
1,519.80
(98.83%)
📈 view plot
🚷 view threshold
2,273.00
(-0.26%)
2,303.05
(98.70%)
📈 view plot
🚷 view threshold
14.00
(+52.38%)
15.70
(89.16%)
📈 view plot
🚷 view threshold
68.00
(+0.81%)
69.97
(97.18%)
client_sv2_setup_connection_serialize📈 view plot
🚷 view threshold
16,202.00
(+0.13%)
16,488.88
(98.26%)
📈 view plot
🚷 view threshold
5,963.00
(-0.01%)
5,980.80
(99.70%)
📈 view plot
🚷 view threshold
8,657.00
(-0.07%)
8,694.49
(99.57%)
📈 view plot
🚷 view threshold
46.00
(+14.51%)
53.79
(85.52%)
📈 view plot
🚷 view threshold
209.00
(-0.03%)
217.08
(96.28%)
client_sv2_setup_connection_serialize_deserialize📈 view plot
🚨 view alert
🚷 view threshold
35,750.00
(+0.55%)
35,745.38
(100.01%)
📈 view plot
🚷 view threshold
14,888.00
(+0.28%)
14,905.42
(99.88%)
📈 view plot
🚷 view threshold
21,875.00
(+0.31%)
21,915.93
(99.81%)
📈 view plot
🚷 view threshold
101.00
(+8.32%)
115.27
(87.62%)
📈 view plot
🚷 view threshold
382.00
(+0.69%)
384.83
(99.27%)
🐰 View full continuous benchmarking report in Bencher

Copy link
Contributor

🐰 Bencher Report

Branchstd_logic
Testbedsv2
Click to view all benchmark results
BenchmarkLatencyBenchmark Result
nanoseconds (ns)
(Result Δ%)
Upper Boundary
nanoseconds (ns)
(Limit %)
client_sv2_handle_message_common📈 view plot
🚷 view threshold
45.32
(+1.03%)
48.39
(93.65%)
client_sv2_handle_message_mining📈 view plot
🚷 view threshold
73.06
(-2.92%)
94.20
(77.56%)
client_sv2_mining_message_submit_standard📈 view plot
🚷 view threshold
14.69
(+0.17%)
14.92
(98.47%)
client_sv2_mining_message_submit_standard_serialize📈 view plot
🚷 view threshold
277.13
(+2.14%)
306.74
(90.35%)
client_sv2_mining_message_submit_standard_serialize_deserialize📈 view plot
🚷 view threshold
615.65
(+4.42%)
761.21
(80.88%)
client_sv2_open_channel📈 view plot
🚷 view threshold
147.85
(+0.20%)
159.79
(92.53%)
client_sv2_open_channel_serialize📈 view plot
🚷 view threshold
286.69
(+0.61%)
316.58
(90.56%)
client_sv2_open_channel_serialize_deserialize📈 view plot
🚷 view threshold
381.73
(+0.43%)
453.73
(84.13%)
client_sv2_setup_connection📈 view plot
🚷 view threshold
169.23
(+3.83%)
206.14
(82.10%)
client_sv2_setup_connection_serialize📈 view plot
🚷 view threshold
498.27
(+4.22%)
690.86
(72.12%)
client_sv2_setup_connection_serialize_deserialize📈 view plot
🚷 view threshold
983.18
(+1.55%)
1,130.93
(86.94%)
🐰 View full continuous benchmarking report in Bencher

Copy link
Contributor

🐰 Bencher Report

Branchstd_logic
Testbedsv1
Click to view all benchmark results
BenchmarkEstimated CyclesBenchmark Result
estimated cycles
(Result Δ%)
Upper Boundary
estimated cycles
(Limit %)
InstructionsBenchmark Result
instructions
(Result Δ%)
Upper Boundary
instructions
(Limit %)
L1 AccessesBenchmark Result
accesses
(Result Δ%)
Upper Boundary
accesses
(Limit %)
L2 AccessesBenchmark Result
accesses
(Result Δ%)
Upper Boundary
accesses
(Limit %)
RAM AccessesBenchmark Result
accesses
(Result Δ%)
Upper Boundary
accesses
(Limit %)
get_authorize📈 view plot
🚷 view threshold
8,391.00
(-0.79%)
8,829.56
(95.03%)
📈 view plot
🚷 view threshold
3,689.00
(-1.61%)
3,889.36
(94.85%)
📈 view plot
🚷 view threshold
5,156.00
(-1.95%)
5,454.73
(94.52%)
📈 view plot
🚷 view threshold
10.00
(+41.44%)
12.11
(82.58%)
📈 view plot
🚷 view threshold
91.00
(+0.67%)
96.16
(94.63%)
get_submit📈 view plot
🚷 view threshold
95,453.00
(+0.01%)
96,157.88
(99.27%)
📈 view plot
🚷 view threshold
59,346.00
(-0.21%)
59,812.89
(99.22%)
📈 view plot
🚷 view threshold
85,218.00
(-0.24%)
85,908.71
(99.20%)
📈 view plot
🚷 view threshold
45.00
(-1.56%)
66.47
(67.70%)
📈 view plot
🚷 view threshold
286.00
(+2.20%)
289.24
(98.88%)
get_subscribe📈 view plot
🚷 view threshold
7,872.00
(-1.83%)
8,380.90
(93.93%)
📈 view plot
🚷 view threshold
2,765.00
(-2.50%)
2,968.42
(93.15%)
📈 view plot
🚷 view threshold
3,847.00
(-2.97%)
4,146.32
(92.78%)
📈 view plot
🚷 view threshold
14.00
(+15.84%)
21.11
(66.33%)
📈 view plot
🚷 view threshold
113.00
(-0.96%)
120.13
(94.06%)
serialize_authorize📈 view plot
🚷 view threshold
12,214.00
(-0.48%)
12,650.73
(96.55%)
📈 view plot
🚷 view threshold
5,266.00
(-1.02%)
5,459.84
(96.45%)
📈 view plot
🚷 view threshold
7,324.00
(-1.30%)
7,617.00
(96.15%)
📈 view plot
🚷 view threshold
12.00
(+34.27%)
15.01
(79.96%)
📈 view plot
🚷 view threshold
138.00
(+0.46%)
143.65
(96.07%)
serialize_deserialize_authorize📈 view plot
🚷 view threshold
24,344.00
(-1.37%)
25,282.15
(96.29%)
📈 view plot
🚷 view threshold
9,838.00
(-0.55%)
10,054.01
(97.85%)
📈 view plot
🚷 view threshold
13,889.00
(-0.55%)
14,207.29
(97.76%)
📈 view plot
🚷 view threshold
33.00
(-5.19%)
44.23
(74.61%)
📈 view plot
🚷 view threshold
294.00
(-2.39%)
316.01
(93.04%)
serialize_deserialize_handle_authorize📈 view plot
🚷 view threshold
30,050.00
(-0.88%)
30,797.60
(97.57%)
📈 view plot
🚷 view threshold
12,015.00
(-0.57%)
12,223.82
(98.29%)
📈 view plot
🚷 view threshold
17,000.00
(-0.66%)
17,313.54
(98.19%)
📈 view plot
🚷 view threshold
62.00
(+11.38%)
68.08
(91.06%)
📈 view plot
🚷 view threshold
364.00
(-1.43%)
382.28
(95.22%)
serialize_deserialize_handle_submit📈 view plot
🚷 view threshold
126,471.00
(-0.00%)
127,200.14
(99.43%)
📈 view plot
🚷 view threshold
73,200.00
(-0.12%)
73,698.62
(99.32%)
📈 view plot
🚷 view threshold
104,911.00
(-0.15%)
105,666.98
(99.28%)
📈 view plot
🚷 view threshold
105.00
(-2.65%)
136.09
(77.15%)
📈 view plot
🚷 view threshold
601.00
(+0.83%)
605.31
(99.29%)
serialize_deserialize_handle_subscribe📈 view plot
🚷 view threshold
27,468.00
(-1.27%)
28,655.43
(95.86%)
📈 view plot
🚷 view threshold
9,584.00
(-0.67%)
9,788.18
(97.91%)
📈 view plot
🚷 view threshold
13,543.00
(-0.77%)
13,846.13
(97.81%)
📈 view plot
🚷 view threshold
69.00
(+8.85%)
75.82
(91.00%)
📈 view plot
🚷 view threshold
388.00
(-2.00%)
417.55
(92.92%)
serialize_deserialize_submit📈 view plot
🚷 view threshold
115,163.00
(-0.07%)
115,995.57
(99.28%)
📈 view plot
🚷 view threshold
68,060.00
(-0.07%)
68,514.35
(99.34%)
📈 view plot
🚷 view threshold
97,653.00
(-0.09%)
98,356.91
(99.28%)
📈 view plot
🚷 view threshold
65.00
(+3.12%)
79.22
(82.05%)
📈 view plot
🚷 view threshold
491.00
(+0.01%)
499.71
(98.26%)
serialize_deserialize_subscribe📈 view plot
🚷 view threshold
22,850.00
(-1.68%)
24,082.36
(94.88%)
📈 view plot
🚷 view threshold
8,143.00
(-0.74%)
8,345.06
(97.58%)
📈 view plot
🚷 view threshold
11,465.00
(-0.79%)
11,754.61
(97.54%)
📈 view plot
🚷 view threshold
37.00
(-1.00%)
44.66
(82.85%)
📈 view plot
🚷 view threshold
320.00
(-2.59%)
349.51
(91.56%)
serialize_submit📈 view plot
🚷 view threshold
99,854.00
(-0.01%)
100,524.90
(99.33%)
📈 view plot
🚷 view threshold
61,408.00
(-0.18%)
61,859.80
(99.27%)
📈 view plot
🚷 view threshold
88,074.00
(-0.22%)
88,758.33
(99.23%)
📈 view plot
🚷 view threshold
53.00
(+12.04%)
66.42
(79.79%)
📈 view plot
🚷 view threshold
329.00
(+1.39%)
333.77
(98.57%)
serialize_subscribe📈 view plot
🚷 view threshold
11,317.00
(-0.70%)
11,761.02
(96.22%)
📈 view plot
🚷 view threshold
4,118.00
(-1.56%)
4,314.89
(95.44%)
📈 view plot
🚷 view threshold
5,707.00
(-1.97%)
6,002.67
(95.07%)
📈 view plot
🚷 view threshold
16.00
(+24.12%)
20.81
(76.88%)
📈 view plot
🚷 view threshold
158.00
(+0.34%)
164.12
(96.27%)
🐰 View full continuous benchmarking report in Bencher

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
protocols Lowest level protocol logic
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants