Adds as_slice and friends to basic slice and vec iterators#1255
Adds as_slice and friends to basic slice and vec iterators#1255lucascool12 wants to merge 1 commit intorayon-rs:mainfrom
as_slice and friends to basic slice and vec iterators#1255Conversation
This commit adds the methods `as_slice`, `as_mut_slice`, `into_slice` and `AsRef<[T]>` on the appropriate iterators for slice iterators and vec iterators. This allows custom producers to use these iterators to decide on custom split points based on the remaining data in the slice.
Do you have an example of code where you would like to use this? Unlike an
That's good, we like to match precedent from the standard library -- although |
I only just noticed this, so this doesn't actually help me accomplish what I'm trying to do. I'm trying to create a parallel iterator of a join like iterator that works like this (assumes both iterators are sorted): impl<I, J> Iterator for Pairs<I, J>
where
I: Iterator<Item = u64>,
J: Iterator<Item = u64>,
{
type Item = (Option<u64>, Option<u64>);
fn next(&mut self) -> Option<Self::Item> {
match (self.left.peek(), self.right.peek()) {
(None, None) => None,
(Some(_), None) => Some((self.left.next(), None)),
(None, Some(_)) => Some((None, self.right.next())),
(Some(left), Some(right)) => match left.cmp(&right) {
Ordering::Equal => Some((self.left.next(), self.right.next())),
Ordering::Less => Some((self.left.next(), None)),
Ordering::Greater => Some((None, self.right.next())),
},
}
}
}After thinking about it for a little while, I noticed that if you split one of the two iterators you must split the other iterator at a specific point, otherwise you produce duplicates. But actually doing all of this with rayon seems impossible currently. Since the producer types of slices and vecs are all private so even if they have an Any help is greatly appreciated. |
|
I can't think of a way to do that with generic impl ParallelIterator for ParallelPairs {
type Item = (Option<u64>, Option<u64>);
fn drive_unindexed<C>(self, consumer: C) -> C::Result
where
C: UnindexedConsumer<Self::Item>
{
rayon::iter::split(self, |pairs| todo!("find your split points and do it"))
.flat_map_iter(|pairs| todo!("convert into the serial Pairs"))
.drive_unindexed(consumer)
}
} |
|
Thanks for the suggestion, I still wanted it to be generic over anything that contains a slice, such as Vec. Regarding this PR, as you said this doesn't really help my use case at all, regardless if anyone thinks this would be beneficial I'll gladly continue working on this PR until it is ready to be merged. |
This commit adds the methods
as_slice,as_mut_slice,into_sliceandAsRef<[T]>on the appropriate iterators for slice iterators and vec iterators.This allows custom producers to use these iterators to decide on custom split points based on the remaining data in the slice.
These impls are based on the std impls on std::slice::Iter, std::slice::IterMut, and std::vec::IntoIter.