Consolidate developer documentation in Sphinx

.coderabbit.yaml

@@ -45,7 +45,7 @@ reviews:
         For public C++ API headers, additionally check:
         - Doxygen documentation for all public functions/classes
         - API changes flagged for docs/ updates
-        - Breaking changes require deprecation warnings and migration guide updates
+        - Consequential changes called out clearly; the C++ API currently has no stability or deprecation guarantee
 
     - path: "cpp/{src,src_prims,include}/**/*.{cu,cuh,cpp,hpp,h}"
       instructions: |
@@ -108,6 +108,6 @@ knowledge_base:
       - "cpp/agents.md"
       - "python/agents.md"
       - "CONTRIBUTING.md"
-      - "wiki/cpp/DEVELOPER_GUIDE.md"
-      - "wiki/python/DEVELOPER_GUIDE.md"
-      - "wiki/python/ESTIMATOR_GUIDE.md"
+      - "docs/source/developer_guide/cpp/development.md"
+      - "docs/source/developer_guide/python/development.md"
+      - "docs/source/developer_guide/python/estimators.md"

.github/CODEOWNERS

@@ -5,11 +5,10 @@
 cpp/               @NVIDIA/cuml-cpp-codeowners
 
 # docs
-/CONTRIBUTING.md @NVIDIA/cuml-python-codeowners
-/README.md       @NVIDIA/cuml-python-codeowners
-/docs/           @NVIDIA/cuml-python-codeowners
-/wiki/           @NVIDIA/cuml-python-codeowners
-/wiki/cpp        @NVIDIA/cuml-cpp-codeowners
+/CONTRIBUTING.md                     @NVIDIA/cuml-python-codeowners
+/README.md                           @NVIDIA/cuml-python-codeowners
+/docs/                               @NVIDIA/cuml-python-codeowners
+/docs/source/developer_guide/cpp/    @NVIDIA/cuml-cpp-codeowners
 
 #python code owners
 python/            @NVIDIA/cuml-python-codeowners

.github/workflows/pr.yaml

@@ -159,7 +159,6 @@ jobs:
           - '!notebooks/**'
           - '!python/**'
           - '!thirdparty/LICENSES/**'
-          - '!wiki/**'
         test_notebooks:
           - '**'
           - '!**/*/agents.md'
@@ -197,7 +196,6 @@ jobs:
           - '!cpp/README.md'
           - '!cpp/header.html'
           - '!thirdparty/LICENSES/**'
-          - '!wiki/**'
         test_python_conda:
           - '**'
           - '!**/*/agents.md'
@@ -240,7 +238,6 @@ jobs:
           - '!img/**'
           - '!notebooks/**'
           - '!thirdparty/LICENSES/**'
-          - '!wiki/**'
         test_python_wheels:
           - '**'
           - '!**/*/agents.md'
@@ -287,7 +284,6 @@ jobs:
           - '!img/**'
           - '!notebooks/**'
           - '!thirdparty/LICENSES/**'
-          - '!wiki/**'
   checks:
     needs: telemetry-setup
     permissions:

.gitignore

@@ -74,6 +74,7 @@ doxygen_check/
 
 ## Doxygen
 cpp/html
+cpp/xml
 cpp/Doxyfile
 
 # clang tooling

CONTRIBUTING.md

@@ -30,8 +30,8 @@ into three categories:
     or [help wanted](https://github.com/NVIDIA/cuml/issues?q=is%3Aissue+is%3Aopen+label%3A%22help+wanted%22) labels
 3. Comment on the issue saying you are going to work on it.
 4. Get familiar with the developer guide relevant for you:
-    * For C++ developers it is available here [DEVELOPER_GUIDE.md](wiki/cpp/DEVELOPER_GUIDE.md)
-    * For Python developers, a [Python DEVELOPER_GUIDE.md](wiki/python/DEVELOPER_GUIDE.md) is available as well.
+    * For Python developers, read the [Python Developer Guide](docs/source/developer_guide/python/development.md) and [Estimator Guide](docs/source/developer_guide/python/estimators.md).
+    * For C++ developers, read the [C++ and CUDA Developer Guide](docs/source/developer_guide/cpp/development.md).
 5. Code! Make sure to update unit tests!
 6. When done, [create your pull request](https://github.com/NVIDIA/cuml/compare).
 7. Verify that CI passes all [status checks](https://help.github.com/articles/about-status-checks/), or fix if needed.

build.sh

@@ -259,7 +259,7 @@ fi
 
 ################################################################################
 # Configure for building all C++ targets
-if completeBuild || hasArg libcuml || hasArg prims || hasArg bench || hasArg prims-bench || hasArg cppdocs || hasArg cpp-mgtests; then
+if completeBuild || hasArg libcuml || hasArg prims || hasArg bench || hasArg prims-bench || hasArg cppdocs || hasArg pydocs || hasArg cpp-mgtests; then
     if (( BUILD_ALL_GPU_ARCH == 0 )); then
         CUML_CMAKE_CUDA_ARCHITECTURES="NATIVE"
         echo "Building for the architecture of the GPU in the system..."
@@ -343,7 +343,9 @@ if (! hasArg --configure-only) && (completeBuild || hasArg libcuml || hasArg pri
       fi
 fi
 
-if (! hasArg --configure-only) && hasArg cppdocs; then
+if (! hasArg --configure-only) && (hasArg cppdocs || hasArg pydocs); then
+    # Sphinx consumes the Doxygen XML through Breathe, so pydocs also needs the
+    # docs_cuml prerequisite when it is invoked on its own.
     cmake --build "${LIBCUML_BUILD_DIR}" --target docs_cuml
 fi
 

ci/build_docs.sh

@@ -1,5 +1,5 @@
 #!/bin/bash
-# SPDX-FileCopyrightText: Copyright (c) 2023-2026, NVIDIA CORPORATION.
+# SPDX-FileCopyrightText: Copyright (c) 2023-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
 # SPDX-License-Identifier: Apache-2.0
 set -euo pipefail
 
@@ -33,18 +33,39 @@ rapids-print-env
 RAPIDS_DOCS_DIR="$(mktemp -d)"
 export RAPIDS_DOCS_DIR
 
-rapids-logger "Build CPP docs"
+rapids-logger "Generate C++ API XML for Breathe"
 pushd cpp
 doxygen Doxyfile.in
-mkdir -p "${RAPIDS_DOCS_DIR}/libcuml/html"
-mv html/* "${RAPIDS_DOCS_DIR}/libcuml/html"
 popd
 
-rapids-logger "Build Python docs"
+rapids-logger "Build the combined Python and C++ Sphinx documentation"
 pushd docs
 sphinx-build -b dirhtml ./source _html -W
 mkdir -p "${RAPIDS_DOCS_DIR}/cuml/html"
 mv _html/* "${RAPIDS_DOCS_DIR}/cuml/html"
 popd
 
+# The publishing workflow still expects the historical libcuml project. Keep
+# that entry point without duplicating API content: publish only a redirect to
+# the version-matched C++ API inside the combined Sphinx site. This uses the
+# already-initialized RAPIDS_VERSION_MAJOR_MINOR, so it is safe under `set -u`.
+LIBCUML_REDIRECT_DIR="${RAPIDS_DOCS_DIR}/libcuml/html"
+CUML_CPP_API_URL="https://docs.nvidia.com/cuml/${RAPIDS_VERSION_MAJOR_MINOR}/developer_guide/cpp/api/"
+mkdir -p "${LIBCUML_REDIRECT_DIR}"
+cat > "${LIBCUML_REDIRECT_DIR}/index.html" <<EOF
+<!doctype html>
+<html lang="en">
+<head>
+  <meta charset="utf-8">
+  <title>cuML C++ API moved</title>
+  <link rel="canonical" href="${CUML_CPP_API_URL}">
+  <meta http-equiv="refresh" content="0; url=${CUML_CPP_API_URL}">
+  <script>window.location.replace("${CUML_CPP_API_URL}" + window.location.hash);</script>
+</head>
+<body>
+  <p>The cuML C++ API reference moved to <a href="${CUML_CPP_API_URL}">the cuML Developer Guide</a>.</p>
+</body>
+</html>
+EOF
+
 RAPIDS_VERSION_NUMBER="${RAPIDS_VERSION_MAJOR_MINOR}" rapids-upload-docs

conda/environments/all_cuda-129_arch-aarch64.yaml

@@ -4,6 +4,7 @@ channels:
 - rapidsai-nightly
 - conda-forge
 dependencies:
+- breathe
 - c-compiler
 - ccache
 - certifi

conda/environments/all_cuda-129_arch-x86_64.yaml

@@ -4,6 +4,7 @@ channels:
 - rapidsai-nightly
 - conda-forge
 dependencies:
+- breathe
 - c-compiler
 - ccache
 - certifi

conda/environments/all_cuda-133_arch-aarch64.yaml

@@ -4,6 +4,7 @@ channels:
 - rapidsai-nightly
 - conda-forge
 dependencies:
+- breathe
 - c-compiler
 - ccache
 - certifi

conda/environments/all_cuda-133_arch-x86_64.yaml

@@ -4,6 +4,7 @@ channels:
 - rapidsai-nightly
 - conda-forge
 dependencies:
+- breathe
 - c-compiler
 - ccache
 - certifi

cpp/Doxyfile.in

@@ -1130,7 +1130,7 @@ IGNORE_PREFIX          =
 # If the GENERATE_HTML tag is set to YES, doxygen will generate HTML output
 # The default value is: YES.
 
-GENERATE_HTML          = YES
+GENERATE_HTML          = NO
 
 # The HTML_OUTPUT tag is used to specify where the HTML docs will be put. If a
 # relative path is entered the value of OUTPUT_DIRECTORY will be put in front of
@@ -2011,7 +2011,7 @@ MAN_LINKS              = NO
 # captures the structure of the code including all documentation.
 # The default value is: NO.
 
-GENERATE_XML           = NO
+GENERATE_XML           = YES
 
 # The XML_OUTPUT tag is used to specify where the XML pages will be put. If a
 # relative path is entered the value of OUTPUT_DIRECTORY will be put in front of

dependencies.yaml

@@ -526,6 +526,7 @@ dependencies:
     common:
       - output_types: [conda, requirements]
         packages:
+          - breathe
           - graphviz
           - ipython
           - ipykernel

docs/README.md

@@ -1,14 +1,26 @@
-# Building Documentation
-## Building locally:
+# Building the documentation
 
-#### [Build and install cuML](../BUILD.md)
+## Build locally
+
+First [build and install cuML](../BUILD.md). Generate Doxygen XML before the
+Sphinx documentation because Breathe reads that XML while rendering the C++ API
+pages:
 
-#### Generate the docs
 ```bash
-bash build.sh cppdocs pydocs
+./build.sh cppdocs pydocs
 ```
 
-#### Once the process finishes, documentation can be found in build/html
+The `pydocs` target automatically generates the Doxygen XML prerequisite, so it
+also works on its own. Naming both targets as above makes the prerequisite
+explicit without generating it twice. Doxygen writes XML under `cpp/xml/`; it
+does not produce a separately published HTML API site. The Sphinx Makefile
+writes the complete documentation, including the C++ API reference, to
+`docs/build/html/`:
+
 ```bash
-xdg-open build/html/api.html
+xdg-open docs/build/html/index.html
+xdg-open docs/build/html/developer_guide/cpp/api/index.html
 ```
+
+CI uses the `dirhtml` builder instead, staging its version of the same Sphinx
+site from `docs/_html/`.

docs/source/conf.py

@@ -16,9 +16,11 @@
 # documentation root, use os.path.abspath to make it absolute, like shown here.
 #
 import datetime
+import glob
 import os
 import sys
 import textwrap
+import xml.etree.ElementTree as ET
 
 from packaging.version import Version
 
@@ -43,6 +45,7 @@
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
 # ones.
 extensions = [
+    "breathe",
     "numpydoc",
     "sphinx.ext.autodoc",
     "sphinx.ext.autosummary",
@@ -58,6 +61,55 @@
     "sphinx_design",
 ]
 
+breathe_projects = {
+    "cuml": os.path.abspath(
+        os.path.join(os.path.dirname(__file__), "../../cpp/xml")
+    )
+}
+breathe_default_project = "cuml"
+
+
+def clean_doxygen_xml(path: str) -> None:
+    # Doxygen 1.9.1 emits concepts and instantiations that Sphinx cannot parse,
+    # duplicates enum IDs, and gives TSNE_INIT::PCA the same C++ target as ML::PCA.
+    for filename in glob.glob(os.path.join(path, "*.xml")):
+        tree = ET.parse(filename)
+        changed = False
+        for section in tree.findall(".//sectiondef"):
+            for member in list(section.findall("memberdef")):
+                type_node = member.find("type")
+                type_text = (
+                    "".join(type_node.itertext())
+                    if type_node is not None
+                    else ""
+                )
+                if type_text in {"concept", "template void"}:
+                    section.remove(member)
+                    changed = True
+                    continue
+
+                if member.get("kind") != "enum":
+                    continue
+                member_id = member.get("id", "")
+                for value in list(member.findall("enumvalue")):
+                    if (
+                        member.findtext("name") == "TSNE_INIT"
+                        and value.findtext("name") == "PCA"
+                    ):
+                        member.remove(value)
+                        changed = True
+                    elif not value.get("id", "").startswith(member_id):
+                        value.set(
+                            "id", f"{member_id}_{value.findtext('name')}"
+                        )
+                        changed = True
+        if changed:
+            tree.write(filename, encoding="UTF-8", xml_declaration=True)
+
+
+for project_path in breathe_projects.values():
+    clean_doxygen_xml(project_path)
+
 ipython_mplbackend = "str"
 
 # Add any paths that contain templates here, relative to this directory.

wiki/BENCHMARK.md → docs/source/developer_guide/benchmarking.md

@@ -12,6 +12,7 @@ The benchmark runner also supports YAML manifests. A manifest is the declarative
 - [Running the benchmarks](#running-the-benchmarks)
 - [Common options](#common-options)
 - [Examples](#examples)
+- [Adding algorithm coverage](#adding-algorithm-coverage)
 - [YAML manifests](#yaml-manifests)
   - [Manifest structure](#top-level-schema)
   - [`suite`](#suite)
@@ -61,25 +62,6 @@ python -m cuml.benchmark \
   --backends cpu
 ```
 
-To run a YAML-defined suite:
-
-```bash
-python -m cuml.benchmark \
-  --config python/cuml/cuml/benchmark/configs/single_gpu.yaml \
-  --profile default \
-  --backends gpu \
-  --csv results.csv
-```
-
-To run the tiny harness-validation manifest:
-
-```bash
-python -m cuml.benchmark \
-  --config python/cuml/cuml/benchmark/configs/test.yaml \
-  --profile default \
-  --backends cpu
-```
-
 ### Standalone mode (from the repository)
 
 From the `python/cuml/cuml/benchmark/` directory, you can run without installing cuML:
@@ -345,6 +327,18 @@ When multiple backends are present, timings are grouped on one row:
 
 CSV output remains available through `--csv`, but it is a flat compatibility export. Prefer JSON for regression tracking and reproducibility.
 
+## Adding algorithm coverage
+
+New algorithms should include benchmark coverage for every applicable
+implementation layer. Add a Python estimator to the registry in
+`python/cuml/cuml/benchmark/algorithms.py` and add or update the appropriate
+manifest. For a new C++ algorithm, add a Google Benchmark case under
+`cpp/bench/sg` and list its source in `cpp/bench/CMakeLists.txt`.
+
+Use benchmarks and profiling for performance-sensitive changes to establish
+baselines and investigate bottlenecks, regressions, and unexpected memory
+behavior.
+
 ## YAML manifests
 
 A manifest defines a benchmark suite, default settings for the suite, and the individual benchmark entries to run.

docs/source/developer_guide/contributing.rst

@@ -0,0 +1,20 @@
+Contributing
+============
+
+Start with the repository's `contribution guidelines
+<https://github.com/NVIDIA/cuml/blob/main/CONTRIBUTING.md>`_ for proposing
+changes, preparing pull requests, running repository checks, and working with
+continuous integration. Those guidelines are the canonical source for the
+contribution process.
+
+Use the implementation-specific sections of this Developer Guide after choosing
+a change:
+
+* :doc:`Python development <python/development>` covers Python style, testing,
+  validation, memory management, and documentation.
+* :doc:`Python estimator development <python/estimators>` describes the
+  ``cuml.Base`` estimator contract and implementation patterns.
+* :doc:`C++ and CUDA development <cpp/index>` covers C++/CUDA source layout,
+  resources, testing, and the internal C++ API reference.
+* :doc:`Benchmarking <benchmarking>` explains the benchmark CLI, manifests, and
+  adding algorithm coverage.