diff --git a/dev/.documenter-siteinfo.json b/dev/.documenter-siteinfo.json index ce71e7b..8b82495 100644 --- a/dev/.documenter-siteinfo.json +++ b/dev/.documenter-siteinfo.json @@ -1 +1 @@ -{"documenter":{"julia_version":"1.9.4","generation_timestamp":"2024-07-31T08:20:50","documenter_version":"1.5.0"}} \ No newline at end of file +{"documenter":{"julia_version":"1.9.4","generation_timestamp":"2024-08-07T08:22:24","documenter_version":"1.5.0"}} \ No newline at end of file diff --git a/dev/MPI/index.html b/dev/MPI/index.html index ffe7598..0962634 100644 --- a/dev/MPI/index.html +++ b/dev/MPI/index.html @@ -34,4 +34,4 @@ MPI.Sendrecv!(v, next_rank, 0, r, prev_rank, 0, MPI.COMM_WORLD) -@assert all(r .== prev_rank)

Internally, the pointer to the data of the view is passed to MPI, there is no copy of the data, regardless of the memory space where the view is stored in.

If Kokkos.span_is_contiguous(view) == true, then the whole memory span of the view is passed to MPI as a single block of data.

For non-contiguous views (such as LayoutStride), a custom MPI.Datatype is built to exactly represent the view.

Support for GPU-awareness should be seamless, as long as your MPI implementation supports the GPU.

+@assert all(r .== prev_rank)

Internally, the pointer to the data of the view is passed to MPI, there is no copy of the data, regardless of the memory space where the view is stored in.

If Kokkos.span_is_contiguous(view) == true, then the whole memory span of the view is passed to MPI as a single block of data.

For non-contiguous views (such as LayoutStride), a custom MPI.Datatype is built to exactly represent the view.

Support for GPU-awareness should be seamless, as long as your MPI implementation supports the GPU.

diff --git a/dev/calling_c/index.html b/dev/calling_c/index.html index cd54c41..1d20090 100644 --- a/dev/calling_c/index.html +++ b/dev/calling_c/index.html @@ -64,4 +64,4 @@ Kokkos.Views.View{Float64, 1, Kokkos.LayoutRight, Kokkos.HostSpace}
Warning

If any view which has been allocated by an external library is owned by Julia (i.e. in the case where it is Julia which should call the destructor), it must be finalized before unloading the library (i.e. either finalize must be called on the view, or the garbage collector did so automatically beforehand).

Failure to do so will result in nasty segfaults when the GC tries to call the finalizer on the view, which also happens when Julia is exiting.

The segfault could look like this:

signal (11): Segmentation error
 in expression starting at /home/Kokkos/test/runtests.jl:19
 unknown function (ip: 0x7f928f488090)
-_ZN6Kokkos4Impl22SharedAllocationRecordIvvE9decrementEPS2_ at /home/Kokkos/.kokkos-build/wrapper-build-release/lib/kokkos/core/src/libkokkoscore.so.4.0 (unknown line)

The main clue that it is a finalizer error is the fact it happens in Kokkos::Impl::SharedAllocationRecord::decrement.

+_ZN6Kokkos4Impl22SharedAllocationRecordIvvE9decrementEPS2_ at /home/Kokkos/.kokkos-build/wrapper-build-release/lib/kokkos/core/src/libkokkoscore.so.4.0 (unknown line)

The main clue that it is a finalizer error is the fact it happens in Kokkos::Impl::SharedAllocationRecord::decrement.

diff --git a/dev/compilation/index.html b/dev/compilation/index.html index 9c86d70..ebf8cdd 100644 --- a/dev/compilation/index.html +++ b/dev/compilation/index.html @@ -7,4 +7,4 @@ kokkos_path = nothing, kokkos_options = nothing, inherit_options = true -)

Construct a new Kokkos project in source_dir built to build_dir using CMake. After compilation, the target library should be found at joinpath(build_dir, target_lib_path).

The shared library extension of target_lib_path can be omitted, as it is added if needed by Libdl.dlopen.

target is the CMake target needed to build the library.

build_type controls the CMAKE_BUILD_TYPE variable, and cmake_options contains all other options passed to each CMake command.

kokkos_path sets the Kokkos_ROOT CMake variable, to be used by find_package.

kokkos_path should be left to nothing in most cases, as it will be replaced by the installation directory of Kokkos correctly configured with the current options (see get_kokkos_install_dir).

Warning

The Kokkos version of the project must match the version used by the internal wrapper library. If this is not the case, the program may silently fail. Use the kokkos_path configuration variable to change the Kokkos version throughout Kokkos.jl.

kokkos_options is a Dict of Kokkos variables needed to configure the project (see the docs). Values of type Bool are converted to "ON" and "OFF", all other types directly are converted to strings. Each variable is then passed to CMake as "-D$(name)=$(value)".

If inherit_options is true, the cmake_options of the Kokkos wrapper project will be appended at the front of the new project's cmake_options. kokkos_options are not inherited here since Kokkos's CMake mechanisms do this automatically.

All commands are invoked from the current working directory.

source
CMakeKokkosProject(project::CMakeKokkosProject, target, target_lib_path)

Construct a project from another, for a different target.

The source and build directories will stay the same, and options will be copied.

source
Kokkos.build_dir — Function
build_dir(project::KokkosProject)

Return the build directory of project.

source
Kokkos.source_dir — Function
source_dir(project::KokkosProject)

Return the source directory of project.

source
Kokkos.lib_path — Function
lib_path(project::KokkosProject)

Return the path to the target library for project.

source
Kokkos.options — Function
options(project::KokkosProject)

Return the set of options to the target library for project.

source
Kokkos.option! — Function
option!(project::KokkosProject, name::String, val; prefix="Kokkos_")

Sets the given Kokkos option for the project to val. This will result in the following compilation option: "-D$(prefix)$(name)=$(val)".

source
Kokkos.configuration_changed — Function
configuration_changed(project::KokkosProject)

Return true if the configuration of project changed, and needs to be recompiled.

source
Kokkos.configuration_changed! — Function
configuration_changed!(project::KokkosProject, val::Bool = true)

Sets the configuration state of project to val.

source
Kokkos.configure — Function
configure(project::KokkosProject)

Configure the project with its current options.

source
Kokkos.compile — Function
compile(project::KokkosProject)

Builds all source files of the project.

If the project's configuration changed, it is reconfigured first.

source
Kokkos.clean — Function
clean(project::KokkosProject; reset=false)

Clean the project, forcing a recompilation of all source files.

If reset == true, the entire build_dir is removed, therefore ensuring that no cached CMake variable can interfere with the build.

source

Utilities

Kokkos.Wrapper.get_jlcxx_root — Function
get_jlcxx_root()

Return the directory where the file "JlCxxConfig.cmake" is located for the currently loaded CxxWrap package.

Setting the CMake variable JlCxx_ROOT to this path allows the CMake function find_package to load JlCxx.

source
Kokkos.Wrapper.get_kokkos_build_dir — Function
get_kokkos_build_dir()

The directory where Kokkos is compiled.

source
Kokkos.Wrapper.get_kokkos_dir — Function
get_kokkos_dir()

The directory where the sources of Kokkos are located.

If KOKKOS_PATH is not set, it defaults to the sources of Kokkos packaged with Kokkos.jl.

This directory is meant to be passed to the CMake function add_subdirectory in order to load Kokkos as an in-tree build.

source
Kokkos.Wrapper.get_kokkos_install_dir — Function
get_kokkos_install_dir()

The directory where Kokkos is installed.

This directory can be passed to the CMake function find_package through the Kokkos_ROOT variable in order to load Kokkos with the same options and backends as the ones used by Kokkos.jl.

source
Kokkos.Wrapper.clean_cmake_files — Function
clean_cmake_files()

Clears the CMake cache of the wrapper library, and removes all build files, as well as compiled function libraries.

source
+)

Construct a new Kokkos project in source_dir built to build_dir using CMake. After compilation, the target library should be found at joinpath(build_dir, target_lib_path).

The shared library extension of target_lib_path can be omitted, as it is added if needed by Libdl.dlopen.

target is the CMake target needed to build the library.

build_type controls the CMAKE_BUILD_TYPE variable, and cmake_options contains all other options passed to each CMake command.

kokkos_path sets the Kokkos_ROOT CMake variable, to be used by find_package.

kokkos_path should be left to nothing in most cases, as it will be replaced by the installation directory of Kokkos correctly configured with the current options (see get_kokkos_install_dir).

Warning

The Kokkos version of the project must match the version used by the internal wrapper library. If this is not the case, the program may silently fail. Use the kokkos_path configuration variable to change the Kokkos version throughout Kokkos.jl.

kokkos_options is a Dict of Kokkos variables needed to configure the project (see the docs). Values of type Bool are converted to "ON" and "OFF", all other types directly are converted to strings. Each variable is then passed to CMake as "-D$(name)=$(value)".

If inherit_options is true, the cmake_options of the Kokkos wrapper project will be appended at the front of the new project's cmake_options. kokkos_options are not inherited here since Kokkos's CMake mechanisms do this automatically.

All commands are invoked from the current working directory.

source
CMakeKokkosProject(project::CMakeKokkosProject, target, target_lib_path)

Construct a project from another, for a different target.

The source and build directories will stay the same, and options will be copied.

source
Kokkos.build_dir — Function
build_dir(project::KokkosProject)

Return the build directory of project.

source
Kokkos.source_dir — Function
source_dir(project::KokkosProject)

Return the source directory of project.

source
Kokkos.lib_path — Function
lib_path(project::KokkosProject)

Return the path to the target library for project.

source
Kokkos.options — Function
options(project::KokkosProject)

Return the set of options to the target library for project.

source
Kokkos.option! — Function
option!(project::KokkosProject, name::String, val; prefix="Kokkos_")

Sets the given Kokkos option for the project to val. This will result in the following compilation option: "-D$(prefix)$(name)=$(val)".

source
Kokkos.configuration_changed — Function
configuration_changed(project::KokkosProject)

Return true if the configuration of project changed, and needs to be recompiled.

source
Kokkos.configuration_changed! — Function
configuration_changed!(project::KokkosProject, val::Bool = true)

Sets the configuration state of project to val.

source
Kokkos.configure — Function
configure(project::KokkosProject)

Configure the project with its current options.

source
Kokkos.compile — Function
compile(project::KokkosProject)

Builds all source files of the project.

If the project's configuration changed, it is reconfigured first.

source
Kokkos.clean — Function
clean(project::KokkosProject; reset=false)

Clean the project, forcing a recompilation of all source files.

If reset == true, the entire build_dir is removed, therefore ensuring that no cached CMake variable can interfere with the build.

source

Utilities

Kokkos.Wrapper.get_jlcxx_root — Function
get_jlcxx_root()

Return the directory where the file "JlCxxConfig.cmake" is located for the currently loaded CxxWrap package.

Setting the CMake variable JlCxx_ROOT to this path allows the CMake function find_package to load JlCxx.

source
Kokkos.Wrapper.get_kokkos_build_dir — Function
get_kokkos_build_dir()

The directory where Kokkos is compiled.

source
Kokkos.Wrapper.get_kokkos_dir — Function
get_kokkos_dir()

The directory where the sources of Kokkos are located.

If KOKKOS_PATH is not set, it defaults to the sources of Kokkos packaged with Kokkos.jl.

This directory is meant to be passed to the CMake function add_subdirectory in order to load Kokkos as an in-tree build.

source
Kokkos.Wrapper.get_kokkos_install_dir — Function
get_kokkos_install_dir()

The directory where Kokkos is installed.

This directory can be passed to the CMake function find_package through the Kokkos_ROOT variable in order to load Kokkos with the same options and backends as the ones used by Kokkos.jl.

source
Kokkos.Wrapper.clean_cmake_files — Function
clean_cmake_files()

Clears the CMake cache of the wrapper library, and removes all build files, as well as compiled function libraries.

source
diff --git a/dev/config_options/index.html b/dev/config_options/index.html index e6ee961..fce2254 100644 --- a/dev/config_options/index.html +++ b/dev/config_options/index.html @@ -1,2 +1,2 @@ -Configuration options · Kokkos.jl

Configuration Options

Configuration options are set using Preferences.jl. Your LocalPreferences.jl file will store the options needed by your current project in a [Kokkos] section.

Important

Unlike some packages using Preferences.jl, it is possible to change all options during the same Julia session, using their setters. However, if you need to dynamically configure Kokkos.jl, it must be done before loading the wrapper library. After calling load_wrapper_lib (or initialize), all options will be locked, and any changes made afterward will not affect the current Julia session.

kokkos_version

The version of Kokkos to use. Must be a valid version tag in the official Kokkos repository (e.g. "4.0.00").

Only used when kokkos_path is not set, and defaults to the one of the packaged sources of kokkos.

Each version is stored in the package's scratch space, which is checked-out upon loading the Kokkos wrapper.

Special versions like latest or 3.7-latest are supported:

  • latest: use the latest release
  • 3-latest: use the latest release of major version 3
  • 4.1-latest: use the latest release of major and minor versions 4 and 1

Can be set using Kokkos.set_kokkos_version(). The value for the current Julia session is stored in Kokkos.LOCAL_KOKKOS_VERSION_STR.

kokkos_path

The path to the Kokkos sources (not an installation!) to use. If not set, it defaults to the Kokkos version packaged with Kokkos.jl.

Can be set using Kokkos.set_kokkos_path(). The value for the current Julia session is stored in Kokkos.KOKKOS_PATH.

cmake_options

The list of CMake options to pass to all CMakeKokkosProjects.

Can be set using Kokkos.set_cmake_options(). The value for the current Julia session is stored in Kokkos.KOKKOS_CMAKE_OPTIONS.

kokkos_options

The list of Kokkos options to pass to all KokkosProjects.

It can be passed as a list of "Kokkos_<option_name>=<value>", or as a Dict{String, Any} (Bool values will be converted to "ON" and "OFF", others to strings).

Can be set using Kokkos.set_kokkos_options(). The value for the current Julia session is stored in Kokkos.KOKKOS_LIB_OPTIONS.

backends

The list of Kokkos backends to compile for. When in uppercase and prefixed by Kokkos_ENABLE_ the names should correspond to one of the valid device backends options. Defaults to Serial and OpenMP.

Can be set using Kokkos.set_backends(), using a vector of String or ExecutionSpace subtypes.

The value for the current Julia session is stored in Kokkos.KOKKOS_BACKENDS.

build_type

CMake build type.

Can be set using Kokkos.set_build_type(). The value for the current Julia session is stored in Kokkos.KOKKOS_BUILD_TYPE.

build_dir

Main building directory for the current session. The wrapping library is built in $(build_dir)/wrapper-build-$(build_type)/.

Can be set using Kokkos.set_build_dir(), Kokkos.build_in_scratch, Kokkos.build_in_tmp or Kokkos.build_in_project. The value for the current Julia session is stored in Kokkos.KOKKOS_BUILD_DIR.

Note

Kokkos.set_build_dir() accepts a kwarg local_only (default: false) which allows to set the build directory without writing to LocalPreferences.toml.

This is only useful in MPI applications, where only one process should modify the confiuration options and compile code. Calling Kokkos.set_build_dir(build_dir; local_only=true) on non-root processes will allow them to find the libraries compiled by the root process.

+Configuration options · Kokkos.jl

Configuration Options

Configuration options are set using Preferences.jl. Your LocalPreferences.jl file will store the options needed by your current project in a [Kokkos] section.

Important

Unlike some packages using Preferences.jl, it is possible to change all options during the same Julia session, using their setters. However, if you need to dynamically configure Kokkos.jl, it must be done before loading the wrapper library. After calling load_wrapper_lib (or initialize), all options will be locked, and any changes made afterward will not affect the current Julia session.

kokkos_version

The version of Kokkos to use. Must be a valid version tag in the official Kokkos repository (e.g. "4.0.00").

Only used when kokkos_path is not set, and defaults to the one of the packaged sources of kokkos.

Each version is stored in the package's scratch space, which is checked-out upon loading the Kokkos wrapper.

Special versions like latest or 3.7-latest are supported:

  • latest: use the latest release
  • 3-latest: use the latest release of major version 3
  • 4.1-latest: use the latest release of major and minor versions 4 and 1

Can be set using Kokkos.set_kokkos_version(). The value for the current Julia session is stored in Kokkos.LOCAL_KOKKOS_VERSION_STR.

kokkos_path

The path to the Kokkos sources (not an installation!) to use. If not set, it defaults to the Kokkos version packaged with Kokkos.jl.

Can be set using Kokkos.set_kokkos_path(). The value for the current Julia session is stored in Kokkos.KOKKOS_PATH.

cmake_options

The list of CMake options to pass to all CMakeKokkosProjects.

Can be set using Kokkos.set_cmake_options(). The value for the current Julia session is stored in Kokkos.KOKKOS_CMAKE_OPTIONS.

kokkos_options

The list of Kokkos options to pass to all KokkosProjects.

It can be passed as a list of "Kokkos_<option_name>=<value>", or as a Dict{String, Any} (Bool values will be converted to "ON" and "OFF", others to strings).

Can be set using Kokkos.set_kokkos_options(). The value for the current Julia session is stored in Kokkos.KOKKOS_LIB_OPTIONS.

backends

The list of Kokkos backends to compile for. When in uppercase and prefixed by Kokkos_ENABLE_ the names should correspond to one of the valid device backends options. Defaults to Serial and OpenMP.

Can be set using Kokkos.set_backends(), using a vector of String or ExecutionSpace subtypes.

The value for the current Julia session is stored in Kokkos.KOKKOS_BACKENDS.

build_type

CMake build type.

Can be set using Kokkos.set_build_type(). The value for the current Julia session is stored in Kokkos.KOKKOS_BUILD_TYPE.

build_dir

Main building directory for the current session. The wrapping library is built in $(build_dir)/wrapper-build-$(build_type)/.

Can be set using Kokkos.set_build_dir(), Kokkos.build_in_scratch, Kokkos.build_in_tmp or Kokkos.build_in_project. The value for the current Julia session is stored in Kokkos.KOKKOS_BUILD_DIR.

Note

Kokkos.set_build_dir() accepts a kwarg local_only (default: false) which allows to set the build directory without writing to LocalPreferences.toml.

This is only useful in MPI applications, where only one process should modify the confiuration options and compile code. Calling Kokkos.set_build_dir(build_dir; local_only=true) on non-root processes will allow them to find the libraries compiled by the root process.

diff --git a/dev/dynamic_compilation/index.html b/dev/dynamic_compilation/index.html index 9aa47fd..ff7a0b7 100644 --- a/dev/dynamic_compilation/index.html +++ b/dev/dynamic_compilation/index.html @@ -6,4 +6,4 @@ function my_program(x) my_method(x) # Will compile my_method(x) -end

Here the second invocation of my_method is still done in the same world in which my_program was invoked, even though the first invocation increased the global world counter. Therefore, we need to ensure method has not been specialized in the latest world before trying to compile.

source
Kokkos.DynamicCompilation.compile_and_load — Function
compile_and_load(current_module, cmake_target; kwargs...)

Check if the library of cmake_target compiled with kwargs exists, if not compile it, then load it.

The library is a CxxWrap module, which is then loaded into current_module in the sub-module Impl<number> with '<number>' the total count of calls to compile_and_load in this Julia session.

source
Kokkos.DynamicCompilation.has_specialization — Function
has_specialization(func, args_t::Tuple{Vararg{Type}})

True if the most specific method of func applicable to args_t has no @nospecialize annotation on any argument.

source
Kokkos.DynamicCompilation.call_more_specific — Function
call_more_specific(func, args)

Call func with args with Base.invokelatest(func, args...), but only if there is a specialized method for the arguments (an error is raised otherwise).

After calling compile_and_load from a @nospecialize method meant to define a new method specialized for args, this function will prevent infinite recursion if the new method is not applicable to args.

source
Kokkos.DynamicCompilation.clean_libs — Function
clean_libs()

Remove all shared libraries generated for compilation on demand of some methods or types.

Libraries are not unloaded, therefore subsequent calls to compile_and_load might not trigger recompilation.

source
Kokkos.DynamicCompilation.compilation_lock — Function
compilation_lock(func)

Asserts that only a single process (and single thread) is compiling at once.

By default, there is only a lock on tasks of the current process.

If MPI.jl is loaded, a PID lock file is also used (see FileWatching.Pidfile, or Pidfile.jl before 1.9). Lock files have the advantage that no collective MPI operations are needed, however they only work if all MPI ranks share the same filesystem, and if this is not the case then there is no need for lock files. This can be overloaded with the JULIA_KOKKOS_USE_COMPILATION_LOCK_FILE environment variable.

Note that it is not the current process' ID that is used in the PID lock file, since PID are not guaranteed to be unique in a MPI application. Instead a random 32-bit number is used, constant for this process.

source
+end

Here the second invocation of my_method is still done in the same world in which my_program was invoked, even though the first invocation increased the global world counter. Therefore, we need to ensure method has not been specialized in the latest world before trying to compile.

source
Kokkos.DynamicCompilation.compile_and_load — Function
compile_and_load(current_module, cmake_target; kwargs...)

Check if the library of cmake_target compiled with kwargs exists, if not compile it, then load it.

The library is a CxxWrap module, which is then loaded into current_module in the sub-module Impl<number> with '<number>' the total count of calls to compile_and_load in this Julia session.

source
Kokkos.DynamicCompilation.has_specialization — Function
has_specialization(func, args_t::Tuple{Vararg{Type}})

True if the most specific method of func applicable to args_t has no @nospecialize annotation on any argument.

source
Kokkos.DynamicCompilation.call_more_specific — Function
call_more_specific(func, args)

Call func with args with Base.invokelatest(func, args...), but only if there is a specialized method for the arguments (an error is raised otherwise).

After calling compile_and_load from a @nospecialize method meant to define a new method specialized for args, this function will prevent infinite recursion if the new method is not applicable to args.

source
Kokkos.DynamicCompilation.clean_libs — Function
clean_libs()

Remove all shared libraries generated for compilation on demand of some methods or types.

Libraries are not unloaded, therefore subsequent calls to compile_and_load might not trigger recompilation.

source
Kokkos.DynamicCompilation.compilation_lock — Function
compilation_lock(func)

Asserts that only a single process (and single thread) is compiling at once.

By default, there is only a lock on tasks of the current process.

If MPI.jl is loaded, a PID lock file is also used (see FileWatching.Pidfile, or Pidfile.jl before 1.9). Lock files have the advantage that no collective MPI operations are needed, however they only work if all MPI ranks share the same filesystem, and if this is not the case then there is no need for lock files. This can be overloaded with the JULIA_KOKKOS_USE_COMPILATION_LOCK_FILE environment variable.

Note that it is not the current process' ID that is used in the PID lock file, since PID are not guaranteed to be unique in a MPI application. Instead a random 32-bit number is used, constant for this process.

source
diff --git a/dev/environment/index.html b/dev/environment/index.html index 14beec8..24a6cf6 100644 --- a/dev/environment/index.html +++ b/dev/environment/index.html @@ -10,4 +10,4 @@ bind = "close", num_threads = Threads.nthreads() )

Helper function to set the main OpenMP environment variables used by Kokkos. It must be called before calling initialize.

places sets OMP_PLACES. bind sets OMP_PROC_BIND. num_threads sets OMP_NUM_THREADS.

Note that since Julia threads and OpenMP threads are decoupled, there is no constraint imposed by Julia on OpenMP threads: there can be as many threads as needed.

Warning

Pinning the Julia threads with ThreadPinning.jl or with the JULIA_EXCLUSIVE environment variable can have an impact on OpenMP thread affinities, making the OpenMP variables useless.

source

Synchronization

Kokkos.fence — Function
fence()
-fence(label::String)

Wait for all asynchronous Kokkos operations to complete.

Equivalent to Kokkos::fence().

source

Constants

Kokkos.KOKKOS_VERSION — Constant
KOKKOS_VERSION::VersionNumber

The Kokkos version currently loaded.

nothing if Kokkos is not yet loaded. See kokkos_version for the version of the packaged installation of Kokkos, which is defined before loading Kokkos.

source
+fence(label::String)

Wait for all asynchronous Kokkos operations to complete.

Equivalent to Kokkos::fence().

source

Constants

Kokkos.KOKKOS_VERSION — Constant
KOKKOS_VERSION::VersionNumber

The Kokkos version currently loaded.

nothing if Kokkos is not yet loaded. See kokkos_version for the version of the packaged installation of Kokkos, which is defined before loading Kokkos.

source
diff --git a/dev/inaccessible_views/index.html b/dev/inaccessible_views/index.html index 19df581..a048122 100644 --- a/dev/inaccessible_views/index.html +++ b/dev/inaccessible_views/index.html @@ -15,4 +15,4 @@ 1 julia> copyto!(v, host_v) -

The strength of this approach is the fact that it will be efficient whatever the device backend is. create_mirror_view will simply return the view passed to it if the memory space is the same, making it effectively a no-op. Same for deep_copy, which is a no-op if both arguments are the same.

+

The strength of this approach is the fact that it will be efficient whatever the device backend is. create_mirror_view will simply return the view passed to it if the memory space is the same, making it effectively a no-op. Same for deep_copy, which is a no-op if both arguments are the same.

diff --git a/dev/index.html b/dev/index.html index e8d6ad0..b93dd97 100644 --- a/dev/index.html +++ b/dev/index.html @@ -1,2 +1,2 @@ -Home · Kokkos.jl

Kokkos.jl

Documentation for Kokkos.

Kokkos.jl allows you to create Kokkos::View instances from Julia, to configure and compile a Kokkos project or load an existing library, and call its functions and the Kokkos kernels it defines.

Kokkos.jl supports all backends of Kokkos.

View inherits the AbstractArray interface of Julia, and can therefore be used as a normal Array. All view accesses are done through calls to Kokkos::View::operator(), and therefore can access CPU or GPU memory seamlessly.

Note

Currently Kokkos.jl does not create Kokkos kernels (e.g. using Kokkos::parallel_for) since it would require automatic Julia to C++ code convertion. You also cannot run Julia code in Kokkos kernels, since Julia cannot be used in threads it doesn't own (or adopted, in Julia 1.9).

+Home · Kokkos.jl

Kokkos.jl

Documentation for Kokkos.

Kokkos.jl allows you to create Kokkos::View instances from Julia, to configure and compile a Kokkos project or load an existing library, and call its functions and the Kokkos kernels it defines.

Kokkos.jl supports all backends of Kokkos.

View inherits the AbstractArray interface of Julia, and can therefore be used as a normal Array. All view accesses are done through calls to Kokkos::View::operator(), and therefore can access CPU or GPU memory seamlessly.

Note

Currently Kokkos.jl does not create Kokkos kernels (e.g. using Kokkos::parallel_for) since it would require automatic Julia to C++ code convertion. You also cannot run Julia code in Kokkos kernels, since Julia cannot be used in threads it doesn't own (or adopted, in Julia 1.9).

diff --git a/dev/interop/index.html b/dev/interop/index.html index c7b00c1..164882a 100644 --- a/dev/interop/index.html +++ b/dev/interop/index.html @@ -25,4 +25,4 @@ 2×2 Kokkos.Views.View{Int64, 2, Kokkos.LayoutStride, Kokkos.CudaSpace}: <inaccessible view> julia> size(sub_A_v), strides(sub_A_v) -((2, 2), (1, 4))

Unlike Kokkos.View, it is possible to perform arithmetic operations on a CuArray from the host, as well as indexing device memory (if permitted by CUDA.allowscalar(true) or CUDA.@allowscalar).

AMDGPU.jl

Views can be converted into ROCArrays using Base.unsafe_wrap:

Base.unsafe_wrap — Method
unsafe_wrap(ROCArray, v::Kokkos.View)

Wrap a Kokkos.View into a ROCArray. The view must be stored in one of the HIP device memory spaces.

Views with row-major layout (LayoutRight) will be transposed with Base.adjoint.

Non-contiguous views (Kokkos.span_is_contiguous(v) == false) cannot be represented as ROCArrays.

Warning

The returned ROCArray does not own the data of the view, which then must stay rooted for the entire lifetime of the ROCArray.

source

And ROCArrays can be converted into views with Kokkos.view_wrap, in the same manner as for CuArrays.

+((2, 2), (1, 4))

Unlike Kokkos.View, it is possible to perform arithmetic operations on a CuArray from the host, as well as indexing device memory (if permitted by CUDA.allowscalar(true) or CUDA.@allowscalar).

AMDGPU.jl

Views can be converted into ROCArrays using Base.unsafe_wrap:

Base.unsafe_wrap — Method
unsafe_wrap(ROCArray, v::Kokkos.View)

Wrap a Kokkos.View into a ROCArray. The view must be stored in one of the HIP device memory spaces.

Views with row-major layout (LayoutRight) will be transposed with Base.adjoint.

Non-contiguous views (Kokkos.span_is_contiguous(v) == false) cannot be represented as ROCArrays.

Warning

The returned ROCArray does not own the data of the view, which then must stay rooted for the entire lifetime of the ROCArray.

source

And ROCArrays can be converted into views with Kokkos.view_wrap, in the same manner as for CuArrays.

diff --git a/dev/library_management/index.html b/dev/library_management/index.html index 6df663b..509dcfb 100644 --- a/dev/library_management/index.html +++ b/dev/library_management/index.html @@ -1,3 +1,3 @@ Library Management · Kokkos.jl

Library Management

Kokkos.handle — Function
handle(lib::CLibrary)

Return the handle (a Ptr{Nothing}) of lib, for use with Libdl.dlsym for example.

If lib is invalid (not loaded), returns C_NULL.

source
Kokkos.get_symbol — Function
get_symbol(lib::CLibrary, symbol::Symbol)

Load the pointer to the given symbol. Symbol pointers are cached: Libdl.dlsym is called only if the symbol is not already in the cache.

source
Kokkos.load_lib — Function
load_lib(lib::Union{String, KokkosProject, CLibrary};
-         flags=Libdl.RTLD_LAZY | Libdl.RTLD_LOCAL)

Open a shared library.

lib can be the path to the shared library, an existing CLibrary or a KokkosProject. If lib is a project, its target is supposed to be compiled and up-to-date.

If the library is already loaded, it is not opened another time: this guarantees that calling Libdl.dlclose once will unload the library from memory, if the library wasn't opened from elsewhere.

source
Kokkos.unload_lib — Function
unload_lib(lib::Union{KokkosProject, CLibrary})

Unload a library. Return true if the library has a valid handle and Libdl.dlclose was called.

Because of the mechanism behind shared library loading, it is not guaranteed that the library is unloaded from memory after this call. is_lib_loaded is more reliable than the return value of this function.

The symbol cache of the library is cleared by this function.

source
Kokkos.is_lib_loaded — Function
is_lib_loaded(lib::Union{KokkosProject, CLibrary})

Return true if the library was previously loaded by load_lib and is still present in memory.

If the full path to lib is still present in Libdl.dllist(), the library is considered to be loaded.

source
+ flags=Libdl.RTLD_LAZY | Libdl.RTLD_LOCAL)

Open a shared library.

lib can be the path to the shared library, an existing CLibrary or a KokkosProject. If lib is a project, its target is supposed to be compiled and up-to-date.

If the library is already loaded, it is not opened another time: this guarantees that calling Libdl.dlclose once will unload the library from memory, if the library wasn't opened from elsewhere.

source
Kokkos.unload_lib — Function
unload_lib(lib::Union{KokkosProject, CLibrary})

Unload a library. Return true if the library has a valid handle and Libdl.dlclose was called.

Because of the mechanism behind shared library loading, it is not guaranteed that the library is unloaded from memory after this call. is_lib_loaded is more reliable than the return value of this function.

The symbol cache of the library is cleared by this function.

source
Kokkos.is_lib_loaded — Function
is_lib_loaded(lib::Union{KokkosProject, CLibrary})

Return true if the library was previously loaded by load_lib and is still present in memory.

If the full path to lib is still present in Libdl.dllist(), the library is considered to be loaded.

source
diff --git a/dev/spaces/index.html b/dev/spaces/index.html index 79d9605..f67efa9 100644 --- a/dev/spaces/index.html +++ b/dev/spaces/index.html @@ -2,4 +2,4 @@ Execution & Memory Spaces · Kokkos.jl

Execution & Memory Spaces

Kokkos.Space — Type
Space

Abstract super type of all execution and memory spaces.

Main subtypes:

  • ExecutionSpaces: super type of all execution spaces
  • MemorySpaces: super type of all memory spaces

All Kokkos spaces have a main abstract type (Serial, Cuda, HostSpace, HIPSpace...) which are defined even if it has not been compiled on the C++ side. Those main abstract types should be the ones used when specifying a space. This allows methods like enabled to work independently from the wrapper library.

When a space is enabled, a sub-type of its main type is defined by CxxWrap, leading to the following type structure: SerialImplAllocated <: SerialImpl <: Serial <: ExecutionSpace <: Space. Below the main space type (here, Serial), the sub-types are only defined if they are enabled, and therefore they should not be relied upon.

Navigating the type tree can be made easier through main_space_type.

source
Kokkos.ExecutionSpace — Type
ExecutionSpace <: Space

Abstract super-type of all execution spaces.

Sub-types:

  • Serial
  • OpenMP
  • OpenACC
  • OpenMPTarget
  • Threads
  • Cuda
  • HIP
  • HPX
  • SYCL

All sub-types are always defined, but only some of them are enabled. To enable an execution space, you must enable its related Kokkos backend, e.g. "-DKokkos_ENABLE_SERIAL=ON" for the Serial execution space.

To do this you can set the backends option with Kokkos.set_backends, or specify the option directly through kokkos_options.

source
Kokkos.MemorySpace — Type
MemorySpace <: Space

Abstract super-type of all memory spaces.

Sub-types:

  • HostSpace
  • CudaSpace
  • CudaHostPinnedSpace
  • CudaUVMSpace
  • HIPSpace
  • HIPHostPinnedSpace
  • HIPManagedSpace

Sub-types work the same as for ExecutionSpace. They can be enabled by enabling their respective backend.

source
Kokkos.array_layout — Method
array_layout(exec_space::Union{<:ExecutionSpace, Type{<:ExecutionSpace}})

Return the default array layout type of the given execution space.

source
Kokkos.enabled — Function
enabled(space::Union{Space, Type{<:Space}})

Return true if the given execution or memory space is enabled.

source
Kokkos.execution_space — Function
execution_space(space::Union{<:MemorySpace, Type{<:MemorySpace}})

Return the execution space associated by default to the given memory space.

source
Kokkos.memory_space — Method
memory_space(space::Union{<:ExecutionSpace, Type{<:ExecutionSpace}})

Return the memory space associated by default to the given execution space.

source
Kokkos.main_space_type — Function
main_space_type(space::Union{<:Space, Type{<:Space}})

Return the main space type of space, e.g. for Serial, SerialImpl or SerialImplAllocated we get Serial.

source
Kokkos.impl_space_type — Function
impl_space_type(::Type{<:Space})

Opposite of main_space_type: from the main space type (Serial, OpenMP, HostSpace...) return the implementation type (SerialImpl, OpenMPImpl, HostSpaceImpl...). The given space must be enabled.

source
Kokkos.kokkos_name — Function
kokkos_name(space::Union{Space, Type{<:Space}})

Return the name of the execution or memory space as defined by Kokkos.

Equivalent to Kokkos::space::name()

source
Kokkos.fence — Method
fence(exec_space::ExecutionSpace)

Wait for all asynchronous tasks operating on this execution space instance to complete.

Equivalent to exec_space.fence().

source

Constants

Kokkos.ENABLED_EXEC_SPACES — Constant
ENABLED_EXEC_SPACES::Tuple{Vararg{Type{<:ExecutionSpace}}}

List of all enabled Kokkos execution spaces.

nothing if Kokkos is not yet loaded.

source
Kokkos.ENABLED_MEM_SPACES — Constant
ENABLED_MEM_SPACES::Tuple{Vararg{Type{<:MemorySpace}}}

List of all enabled Kokkos memory spaces.

nothing if Kokkos is not yet loaded.

source

Default spaces

Kokkos.DEFAULT_DEVICE_SPACE — Constant
DEFAULT_DEVICE_SPACE::Type{<:ExecutionSpace}

The default execution space where kernels are applied on the device.

Equivalent to Kokkos::DefaultExecutionSpace.

nothing if Kokkos is not yet loaded.

source
Kokkos.DEFAULT_DEVICE_MEM_SPACE — Constant
DEFAULT_DEVICE_MEM_SPACE::Type{<:MemorySpace}

The default memory space where views are stored on the device.

Equivalent to Kokkos::DefaultExecutionSpace::memory_space.

nothing if Kokkos is not yet loaded.

source
Kokkos.DEFAULT_HOST_SPACE — Constant
DEFAULT_HOST_SPACE::Type{<:ExecutionSpace}

The default execution space where kernels are applied on the host.

Equivalent to Kokkos::DefaultHostExecutionSpace.

nothing if Kokkos is not yet loaded.

source
Kokkos.DEFAULT_HOST_MEM_SPACE — Constant
DEFAULT_HOST_MEM_SPACE::Type{<:MemorySpace}

The default memory space where views are stored on the host.

Equivalent to Kokkos::DefaultHostExecutionSpace::memory_space.

nothing if Kokkos is not yet loaded.

source
Kokkos.SHARED_MEMORY_SPACE — Constant
SHARED_MEMORY_SPACE::Union{Nothing, Type{<:MemorySpace}}

The shared memory space between the host and device, or nothing if there is none.

Equivalent to Kokkos::SharedSpace if Kokkos::has_shared_space is true.

nothing if Kokkos is not yet loaded.

source
Kokkos.SHARED_HOST_PINNED_MEMORY_SPACE — Constant
SHARED_HOST_PINNED_MEMORY_SPACE::Union{Nothing, Type{<:MemorySpace}}

The shared pinned memory space between the host and device, or nothing if there is none.

Equivalent to Kokkos::SharedHostPinnedSpace if Kokkos::has_shared_host_pinned_space is true.

nothing if Kokkos is not yet loaded.

source

Backend-specific methods

Those unexported methods are defined in the Kokkos.BackendFunctions module. They have methods only if their respective backend is enabled and Kokkos is initialized.

OpenMP

Some functions of the OpenMP runtime are made available through Kokkos.jl, mostly for debugging purposes and tracking thread affinity.

Cuda / HIP

Kokkos.device_id — Function
device_id([space::Kokkos.Cuda])
 device_id([space::Kokkos.HIP])

Return the ID of the device associated with the given space. If space is not given, the ID of the default device used by Kokkos is returned.

The ID is a 0-index in the list of available devices (as used by cudaGetDeviceProperties or hipGetDeviceProperties for example).

Equivalent to Kokkos::Cuda().cuda_device() or Kokkos::HIP().hip_device().

source
Kokkos.BackendFunctions.wrap_stream — Function
wrap_stream(cuda_stream::Ptr{Cvoid})::Kokkos.Cuda
 wrap_stream(hip_stream::Ptr{Cvoid})::Kokkos.HIP

Return a Kokkos.Cuda or Kokkos.HIP execution space operating on the given stream (a pointer to a cudaStream_t or hipStream_t respectively). Kokkos does not take ownership of the stream.

Equivalent to Kokkos::Cuda(cuda_stream) or Kokkos::HIP(hip_stream).

source
Kokkos.BackendFunctions.stream_ptr — Function
stream_ptr(space::Kokkos.Cuda)
-stream_ptr(space::Kokkos.HIP)

Return the cudaStream_t or hipStream_t of the space as a Ptr{Cvoid}.

Equivalent to Kokkos::Cuda().cuda_stream() or Kokkos::HIP().hip_stream().

source
+stream_ptr(space::Kokkos.HIP)

Return the cudaStream_t or hipStream_t of the space as a Ptr{Cvoid}.

Equivalent to Kokkos::Cuda().cuda_stream() or Kokkos::HIP().hip_stream().

source
Kokkos.BackendFunctions.memory_info — Function
memory_info()

Return (free_memory, total_memory), in bytes, for the current active device.

Equivalent to cuMemGetInfo_v2 or hipMemGetInfo.

source
diff --git a/dev/views/index.html b/dev/views/index.html index 9b06774..a8e52cb 100644 --- a/dev/views/index.html +++ b/dev/views/index.html @@ -67,4 +67,4 @@ v = Kokkos.View{Float64}(undef, 7, 11; layout=LayoutStride(1, 11)) # A 7×11 matrix with row-major layout -v = Kokkos.View{Float64}(undef, 7, 11; layout=LayoutStride(7, 1))

This differs slightly from the C++ Kokkos constructor, where dimensions and strides are interleaved.

source

Constants

Kokkos.Idx — Constant
Idx::Type{<:Integer}

Integer type used by views for indexing on the default execution space. Usually either Cint or Clonglong: a 32bit or 64bit signed integer.

Equivalent to Kokkos::RangePolicy<>::index_type.

nothing if Kokkos is not yet loaded.

source
+v = Kokkos.View{Float64}(undef, 7, 11; layout=LayoutStride(7, 1))

This differs slightly from the C++ Kokkos constructor, where dimensions and strides are interleaved.

source

Constants

Kokkos.Idx — Constant
Idx::Type{<:Integer}

Integer type used by views for indexing on the default execution space. Usually either Cint or Clonglong: a 32bit or 64bit signed integer.

Equivalent to Kokkos::RangePolicy<>::index_type.

nothing if Kokkos is not yet loaded.

source