-
Notifications
You must be signed in to change notification settings - Fork 122
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Refactor: codec-sv2
crate
#1129
Comments
we should seriously consider merging perhaps also |
why? |
These crates are closely linked. To my understanding, you never use one with out the other. For this reason, we should combine them. Currently with them separate, you constantly have to cross reference between these two crates. Unless there is a reason to not combine these two crates, we should plan on this refactor. @Fi3, is there any reason not to combine that crates that you know of? |
I just don't see an advantage that justify the work of putting them together. What is this advantage? Why we should spend time doing that? |
Now is the time where we are really scrutinizing these crates to solidify them moving forward. Can you help me understand by giving me an example of when one of these crates would be used without the other? |
We need to make sure the low-level APIs are lean and easy to use. Right now, they are extremely convoluted. There's no way to understand how to use one single crate without looking for references in multiple other crates. Maybe this is a blindspot for you @Fi3 , because the APIs are fresh in your head and you can navigate them easily (since you wrote most of them). But for us, even while just documenting these crates, it has been extremely challenging to understand how everything is meant to be used together. Our mission here is to make sure the codebase can be easily used by the entire ecosystem, not just one single pool. |
IMHO is lot easier to make a crate that export everything and is well documented rather then refactor all the crates |
https://github.com/demand-open-source/share-accounting-ext/blob/master/Cargo.toml#L15C22-L15C23 here for example I need framing but not codec |
also low level crates are not supposed to be used but we should use an higher level library designed to be easy to use ecc ecc. The low level crates should be used only in special occasion where you really need them, for example for ffi |
Ok, I will take a look at that extension and dig deeper into the code. Thank you. |
ext is just an example main point is IMHO is lot easier to make a crate that export everything and is well documented rather then refactor all the crates |
I dont have a good input about merging/not merging. I will only share that as a new dev in the team, whenever I saw a crate under |
having several low level crates have the main advantage of keeping the lib as small as possible and easier to test and to review. framing-sv2 is very small and I agree that could be merged, I would put it in binary-sv2 not in codec. I also think that is more than fine like that, and that we shouldn't waste our time and energy in merging it. |
I can definitely entertain @Fi3's argument about effort. Whenever we start planning the scope how we will mitigate technical debt, I fully agree that it is very important for us to evaluate the trade-offs of all proposed changes. And for sure, there will absolutely be cases where the benefits will not justify the efforts. I'm not saying this will be the case here. I don't have a clear perspective around that yet, especially since we still have not fully studied the entire scope of To be honest, this discussion is still somewhat premature (which is my fault, since I brought up this topic). After we cover 100% What I said about potentially merging It's actually quite interesting that @Fi3 said that it would probably be better to merge For now, at least in my mind, all options are on the table. The main point is that it is very important that we do our best to learn Or maybe not. Time will tell. 🧘 |
on the top issue description @rrybarczyk suggested:
I would suggest the following naming:
#[cfg(feature = "noise_sv2")]
pub type Encoder<T> = EncryptedEncoder<T>;
#[cfg(not(feature = "noise_sv2"))] // redundant, but written to make it explicitly clear
pub type Encoder<T> = PlainEncoder<T>;
pub struct EncryptedEncoder<T> {
noise_buffer: Buffer,
sv2_buffer: Buffer,
frame: PhantomData<T>,
}
pub struct PlainEncoder<T> {
buffer: Vec<u8>,
frame: PhantomData<T>,
}
#[cfg(feature = "noise_sv2")]
pub type Decoder<B, T> = EncryptedDecoder<B, T>;
#[cfg(not(feature = "noise_sv2"))] // redundant, but written to make it explicitly clear
pub type Decoder<B, T> = PlainDecoder<B, T>;
pub struct EncryptedDecoder<B, T> {
frame: PhantomData<T>,
missing_noise_b: usize,
noise_buffer: B,
sv2_buffer: B,
}
struct PlainDecoder<B, T> {
frame: PhantomData<T>,
missing_b: usize,
buffer: B,
} It's not necessarily the final code organization, just some pseudo-rust to convey the idea around the keywords:
|
Background
This task is an outcome of the
protocols
Rust docs issues tracked in #845.While documenting
protocols::v2::codec-sv2
in #1012, areas of potential code debt were identified. This issue servers as a place to list out these items to then be addressed in an organized manner. The initial Rust documentation effort was an immediate action, while a refactoring (which implies breaking API changes) is not so urgent priority, so for now we should leave this in the backlog for an appropriate moment in the future.Identified Potential Code Debt
Error Variant Naming
Suggestions for
codec_sv2::error::Error
and::CError
refactor:noise_sv2
crate directly in theError
variants likeAeadError(noise_sv2::AeadError)
andNoiseSv2Error(noise_sv2::Error)
.framing_sv2
crate directly in theError
variant likeFramingSv2Error(framing_sv2::Error)
.framing_sv2::Error
variant:FramingError
andFramingSv2Error
, remove one of these variant and handle any downstream effects.codec_sv2::Error
andcodec_sv2::CError
enums):AeadError
-> Either keep as is or do:NoiseSv2Aead
orNoiseAead
BinarySv2Error
->BinarySv2
orBinary
FramingSv2Error
/FramingError
->FramingSv2
orFraming
NoiseSv2Error
->Noise
Error
enums, and the order of theimpl From<external-error> for Error
Uniform Spelling
HandShake
andHandshake
. Everything should beHandshake
with a lowercase "s" (this is how it is defined in The Noise Protocol Framework) . This impacts:codec_sv2::
:lib::State::HandShake
->lib::State::Handshake
error::Error::NotInHandShake
->error::Error::NotInHandshake
error::CError::NotInHandShake
->error::CError::NotInHandshake
framing_sv2::framing2::
(there may be more inframing_sv2
, but these are what I have noticed so far):HandShakeFrame
->HandshakeFrame
EitherFrame::HandShake
->EitherFrame::Handshake
decoder::WithNoise::decode_noise_frame
and thenencoder::NoiseEncoder::encode
. Would it make sense to make these methods uniform likedecoder::WithNoise::decode
andencoder::NoiseEncoder::encode
?decoder::WithNoise
anddecoder::WithoutNoise
. Then we haveencoder::NoiseEncoder
andencoder::Encoder
. Should we make itdecoder::WithNoise
,decoder::WithoutNoise
,encoder::WithNoise
, andencoder::WithoutNoise
? Ordecoder::NoiseDecoder
,decoder::Decoder
,encoder::NoiseEncoder
andencoder::Encoder
?Imports
codec_sv2::lib
, I am not a big fan of importingframing_sv2::framing2::handshake_message_to_frame as h2f
. I think we should importframing_sv2::framing2
then useframing2::handshake_message_to_frame
directly. The upside is it is clearer to the reader, the downside is it is more verbose.use core::cmp
is imported incodec_sv2::error
with the#[cfg(test)]
declarator, but when runningcargo test
I get the following warning saying it is not used. Should it be removed?codec_sv2::decoder
, some imports can be combined into one line. For example,use binary_sv2::{GetSize, Serialize}
.Use of
Result
typesThe following methods use
core::result::Result
but should usecodec_sv2::Result
(unless there is a reason I am not seeing that requires usingcore::result::Result
):State::step_0
State::step_1
State::step_2
Uniform Struct Fields
decoder::WithNoise
andencoder::WithoutNoise
have some shared fields, these fields should appear first and be listed in the same order in each struct.Misc
error
module importsuse core::cmp
under the#[cfg(test)]
but it is unused.decoder.rs
module the typesStandardEitherFrame
andStandardSv2Frame
are defined, however these frames can represent an encoded OR decoded Sv2 frame. I think we should move them out of thedecoder
module into a more appropriate place.encoder.rs
module, we havetype Slice = buffer_sv2::Slice;
with the#[cfg(feature = "with_buffer_pool")]
decorator. Why do we need this type if it is simply assigned tobuffer_sv2::Slice
? Could we just directly usebuffer_sv2::Slice
?encoder.rs
, we have thetype Slice = Vec<u8>
commented out. Can this be completely removed?new
methods should always come first (exception is ifdefault
is present, which it is not for this crate) in animpl
. Seeimpl<T: Serialize + GetSize> Encoder<T>
inencoder
.The text was updated successfully, but these errors were encountered: