Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add slotvec, slotbkvec and slotdeque three containers #21

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

cavivie
Copy link

@cavivie cavivie commented Jun 24, 2024

A slot vec SlotVec is a linear container similar to a vector but the indices are invalidated when the element is removed, and can not incidentally refer to a valid element again even if new elements are pushed. The slot vec implementation relies solely on the index of its elements to index data. In particular, the elements here are Option, which means that Some/None can indicate whether the element is removed or not, respectively, without the need for dedicated slots to store this status information.

A slot bucket vec SlotBkVec is a double-layer liner container similar to a Vec<SlotVec> but the indices are invalidated when the element is removed. The outer layer represents the bucket of slotvec, and the inner layer is the vector that stores the actual element data. The index returned by the public API of a slot bucket vec is not directly the index generated by the internal SlotVec, but the generated index is masked, which causes the index to carry bucket/slot information. When getting an element, you don't need to specify the slot to get its element, but you must specify the bucket parameter when inserting an element. This is by design.

A slot deque SlotDeque is a liner container similar to a vector-deque (VecDeque) but the indices are invalidated when the element is removed. However, since the index of a std/alloc::VecDeque may become invalid due to certain operations (that is, the index assigned in the past will no longer point to the data inserted in the past in the future), we mainly solve this problem here so that we can use the assigned but not removed index to consistently access the elements at the time of insertion without worrying about the impact of push operations. It is simulated by SliceBkVec, which is essentially two SlotVecs, one for the front slice and the other for the back slice.

Thanks to the encapsulation of SlotVec, we can simply replace the current SocketSet in smoltcp and hide some details in SocketSet in SlotVec. I also plan to do something with SocketSet, because SocketSet is currently too simple to meet some of my specific needs. I hope that smoltcp can use user-defined Set to complete some complex scenarios. For example, we simply abstract SocketSet into a trait. This trait contains the necessary operations when smoltcp stack processes socket data. The rest of the operations are extended by the user, such as how to add a socket, because adding a socket and traversing the socket have an extremely subtle relationship, which determines the order of data processing or more to some extent. Of course, these are things to be discussed after this PR.

UPD:
A PoC for replacing SocketSet: cavivie/smoltcp@952aa02

@whitequark
Copy link
Contributor

Thanks for the PR! It looks like a lot of work went into making it. I may not be able to review it immediately but I've put it on my list.

@cavivie
Copy link
Author

cavivie commented Jun 24, 2024

Thanks for your reply, I have updated the specific description, if you have time, you can simply read it, thank you again.

@whitequark
Copy link
Contributor

I appreciate the description, but I would also want to understand the implementation before I sign off with my review.

@cavivie cavivie marked this pull request as draft June 27, 2024 14:10
@cavivie cavivie force-pushed the master branch 3 times, most recently from f9a2789 to e1ebd08 Compare July 1, 2024 10:46
@cavivie cavivie marked this pull request as ready for review July 1, 2024 11:42
@cavivie
Copy link
Author

cavivie commented Jul 1, 2024

This PR has too many lines of code, which may affect your review. But they are related, slotvec depends on mangedslice, slotbkvec depends on slotvec, and slotdeque depends on slotbkvec.

Do you need me to split it so that you can better review this PR?

@whitequark
Copy link
Contributor

Splitting this PR would definitely make the review faster. But it does carry the hazard that an incomplete version will be checked in for a while.

We can agree that I only merge all three parts once they're all reviewed though.

chore: add Debug trait impl for slotvec, slotbkvec and slotdeque

chore: make lifetime for iterators bound more clarity

fix: calculation method both for bucket bits in std/nod_std
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants