Customization point for iterators to be passed directly

[masterleinad]

#855 provides an example were users want to provide their own iterators that reference data on the device. Currently, this causes segfaults since oneDPL is creating a CPU-side copy of the respective elements before launching the algorithm.
This problem also appeared in Kokkos, see kokkos/kokkos#7502. Kokkos defines a RandomAccessIterator class that references a Kokkos::View that manges the respective data and has similar functionality to mdspan (the latter was inspired by the former). This means that the access pattern can be anything although in most cases consecutive elements are a constant stride apart. This means that simply passing raw pointers to oneDPL doesn't represent the iterator space correctly and oneDPL algorithms can't be used (without copying data to a continuous buffer on the host or device).

Whether to create CPU-side buffers is controlled by oneapi::dpl::__internal::is_passed_directly which basically looks for an is_passed_directly alias in the iterator class.
This works but pollutes user-defined classes (and the alias name doesn't indicate that the alias is used in oneDPL).

#861 provides a solution that provides a wrapper class called direct_iterator so that forwards all iterator calls the wrapped class. A different solution is opposed in #861 (comment) by introducing a customization point in the public onedpl namespace such as onedpl::is_passed_directly similar to sycl::is_device_copyable.

[masterleinad]

For Kokkos, it doesn't make much of a difference if a wrapper class or a type trait is used. We need to wrap a user-provided comparator anyway to declare it as device-copyable.

[danhoeflinger]

but it would be much better experience if it was sufficient to add something to the type definition that would be automatically recognized by sycl::is_device_copyable.

To some extent we have this already. It is the is_passed_directly trait defined in the user's type we call the legacy "is_passed_directly" trait (What SYCLomatic currently uses). If we wanted to add a different one to include ONEDPL in the name or something to make it more clear, we could add this into the spec / docs, and add it to our infrastructure.

[masterleinad]

I would be opposed to require an alias (as it is done right now) since it pollutes the respective class with information that is only relevant for interoperability with oneDPL.

[danhoeflinger]

I don't see a reason we could not support both options, a public specialization similar to sycl::is_device_copyable and an alias in the user's type. If either is defined to true_type then we could treat it accordingly. Users could pick what works best for them. @masterleinad, In your case you could ignore that it exists, and use the oneapi::dpl::is_passed_directly specialization for your type.

[masterleinad]

If either is defined to true_type then we could treat it accordingly.

Sure, it just needs to be clear (and tested) what happens in all four cases.

[danhoeflinger]

I agree, it needs to be clearly specified and tested.

If we choose to instead just go with one, at this point, I'd prefer the oneapi::dpl::is_passed_directly specializations because I don't think the downsides will be as severe as see with sycl::is_device_copyable for the reasons discussed above.

[danhoeflinger]

@akukanov @masterleinad
I've provided an initial draft here #1999. I think it seems to be a good fit for what we have described here. I think its best to go for a single solution rather than offer multiple options. After some investigation, I agree with @akukanov that the customization point route is the cleanest option as compared to the other options floated.

[akukanov]

The specification patch: uxlfoundation/oneAPI-spec#620
The implementation: #2126

[akukanov]

Implemented in https://github.com/uxlfoundation/oneDPL/releases/tag/oneDPL-2022.9.0-release