Improve make.jl (#424)

* Improve usability of make.jl
And simplify some links.
* add changelog entry.
* bump CI & docs to also run on 1.11
* Bump version on quarto as well.

.github/workflows/ci.yml

@@ -11,7 +11,7 @@ jobs:
     runs-on: ${{ matrix.os }}
     strategy:
       matrix:
-        julia-version: ["1.6", "1.9", "1.10"]
+        julia-version: ["1.6", "1.10", "1.11"]
         os: [ubuntu-latest, macOS-latest, windows-latest]
     steps:
       - uses: actions/checkout@v4

.github/workflows/documenter.yml

@@ -16,7 +16,7 @@ jobs:
           version: "1.3.353"
       - uses: julia-actions/setup-julia@latest
         with:
-          version: "1.10"
+          version: "1.11"
       - name: Julia Cache
         uses: julia-actions/cache@v2
       - name: Cache Quarto

Changelog.md

@@ -5,6 +5,12 @@ All notable Changes to the Julia package `Manopt.jl` will be documented in this
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.5.4] - unreleased
+
+### Added
+ * An automated detection whether the tutorials are present
+   if not an also no quarto run is done, an automated `--exlcude-tutorials` option is added.
+
 ## [0.5.3] – October 18, 2024
 
 ### Added

docs/make.jl

@@ -11,25 +11,74 @@ Render the `Manopt.jl` documentation with optional arguments
 
 Arguments
 * `--exclude-tutorials` - exclude the tutorials from the menu of Documenter,
-  this can be used if you do not have Quarto installed to still be able to render the docs
-  locally on this machine. This option should not be set on CI.
+  This can be used if not all tutorials are rendered and you want to therefore exclude links
+  to these, especially the corresponding menu. This option should not be set on CI.
+  Locally this is also set if `--quarto` is not set and not all tutorials are rendered.
 * `--help`              - print this help and exit without rendering the documentation
-* `--prettyurls`        – toggle the prettyurls part to true (which is otherwise only true on CI)
-* `--quarto`            – run the Quarto notebooks from the `tutorials/` folder before generating the documentation
-  this has to be run locally at least once for the `tutorials/*.md` files to exist that are included in
-  the documentation (see `--exclude-tutorials`) for the alternative.
-  If they are generated once they are cached accordingly.
+* `--prettyurls`        – toggle the pretty urls part to true, which is always set on CI
+* `--quarto`            – (re)run the Quarto notebooks from the `tutorials/` folder before
+  generating the documentation. If they are generated once they are cached accordingly.
   Then you can spare time in the rendering by not passing this argument.
   If quarto is not run, some tutorials are generated as empty files, since they
-  are referenced from within the documentation. These are currently
-  `Optimize.md` and `ImplementOwnManifold.md`.
-""",
+  are referenced from within the documentation.
+  These are currently `Optimize.md` and `ImplementOwnManifold.md`.
+"""
     )
     exit(0)
 end
 
+run_quarto = "--quarto" in ARGS
+run_on_CI = (get(ENV, "CI", nothing) == "true")
+tutorials_in_menu = !("--exclude-tutorials" ∈ ARGS)
 #
-# (a) if docs is not the current active environment, switch to it
+#
+# (a) setup the tutorials menu – check whether all files exist
+tutorials_menu =
+    "How to..." => [
+        "🏔️ Get started: optimize." => "tutorials/Optimize.md",
+        "Speedup using in-place computations" => "tutorials/InplaceGradient.md",
+        "Use automatic differentiation" => "tutorials/AutomaticDifferentiation.md",
+        "Define objectives in the embedding" => "tutorials/EmbeddingObjectives.md",
+        "Count and use a cache" => "tutorials/CountAndCache.md",
+        "Print debug output" => "tutorials/HowToDebug.md",
+        "Record values" => "tutorials/HowToRecord.md",
+        "Implement a solver" => "tutorials/ImplementASolver.md",
+        "Optimize on your own manifold" => "tutorials/ImplementOwnManifold.md",
+        "Do constrained optimization" => "tutorials/ConstrainedOptimization.md",
+        "Do geodesic regression" => "tutorials/GeodesicRegression.md",
+    ]
+# Check whether all tutorials are rendered, issue a warning if not (and quarto if not set)
+all_tutorials_exist = true
+for (name, file) in tutorials_menu.second
+    fn = joinpath(@__DIR__, "src/", file)
+    if !isfile(fn) || filesize(fn) == 0 # nonexistent or empty file
+        global all_tutorials_exist = false
+        if !run_quarto
+            @warn "Tutorial $name does not exist at $fn."
+            if (!isfile(fn)) &&
+                (endswith(file, "Optimize.md") || endswith(file, "ImplementOwnManifold.md"))
+                @warn "Generating empty file, since this tutorial is linked to from the documentation."
+                touch(fn)
+            end
+        end
+    end
+end
+if !all_tutorials_exist && !run_quarto && !run_on_CI
+    @warn """
+        Not all tutorials exist. Run `make.jl --quarto` to generate them. For this run they are excluded from the menu.
+    """
+    tutorials_in_menu = false
+end
+if !tutorials_in_menu
+    @warn """
+    You are either explicitly or implicitly excluding the tutorials from the documentation.
+    You will not be able to see their menu entries nor their rendered pages.
+    """
+    run_on_CI &&
+        (@error "On CI, the tutorials have to be either rendered with Quarto or be cached.")
+end
+#
+# (b) if docs is not the current active environment, switch to it
 # (from https://github.com/JuliaIO/HDF5.jl/pull/1020/) 
 if Base.active_project() != joinpath(@__DIR__, "Project.toml")
     using Pkg
@@ -39,8 +88,8 @@ if Base.active_project() != joinpath(@__DIR__, "Project.toml")
     Pkg.instantiate()
 end
 
-# (b) Did someone say render?
-if "--quarto" ∈ ARGS
+# (b) If quarto is set, or we are on CI, run quarto
+if run_quarto || run_on_CI
     using CondaPkg
     CondaPkg.withenv() do
         @info "Rendering Quarto"
@@ -55,20 +104,6 @@ if "--quarto" ∈ ARGS
         Pkg.activate(@__DIR__) # but return to the docs one before
         run(`quarto render $(tutorials_folder)`)
     end
-else # fallback to at least create empty files for Optimize and Implement
-    touch(joinpath(@__DIR__, "src/tutorials/Optimize.md"))
-    touch(joinpath(@__DIR__, "src/tutorials/ImplementOwnManifold.md"))
-end
-
-tutorials_in_menu = true
-if "--exclude-tutorials" ∈ ARGS
-    @warn """
-    You are excluding the tutorials from the Menu,
-    which might be done if you can not render them locally.
-
-    Remember that this should never be done on CI for the full documentation.
-    """
-    tutorials_in_menu = false
 end
 
 # (c) load necessary packages for the docs
@@ -101,20 +136,6 @@ for (md_file, doc_file) in
 end
 
 ## Build tutorials menu
-tutorials_menu =
-    "How to..." => [
-        "🏔️ Get started: optimize." => "tutorials/Optimize.md",
-        "Speedup using in-place computations" => "tutorials/InplaceGradient.md",
-        "Use automatic differentiation" => "tutorials/AutomaticDifferentiation.md",
-        "Define objectives in the embedding" => "tutorials/EmbeddingObjectives.md",
-        "Count and use a cache" => "tutorials/CountAndCache.md",
-        "Print debug output" => "tutorials/HowToDebug.md",
-        "Record values" => "tutorials/HowToRecord.md",
-        "Implement a solver" => "tutorials/ImplementASolver.md",
-        "Optimize on your own manifold" => "tutorials/ImplementOwnManifold.md",
-        "Do constrained optimization" => "tutorials/ConstrainedOptimization.md",
-        "Do geodesic regression" => "tutorials/GeodesicRegression.md",
-    ]
 # (e) finally make docs
 bib = CitationBibliography(joinpath(@__DIR__, "src", "references.bib"); style=:alpha)
 links = InterLinks(
@@ -123,7 +144,7 @@ links = InterLinks(
 )
 makedocs(;
     format=Documenter.HTML(;
-        prettyurls=(get(ENV, "CI", nothing) == "true") || ("--prettyurls" ∈ ARGS),
+        prettyurls=run_on_CI || ("--prettyurls" ∈ ARGS),
         assets=["assets/favicon.ico", "assets/citations.css"],
         size_threshold_warn=200 * 2^10, # raise slightly from 100 to 200 KiB
         size_threshold=300 * 2^10,      # raise slightly 200 to to 300 KiB

docs/src/solvers/cma_es.md

@@ -37,7 +37,7 @@ The [`cma_es`](@ref) solver requires the following functions of a manifold to be
 * A [`retract!`](@extref ManifoldsBase :doc:`retractions`)`(M, q, p, X)`; it is recommended to set the [`default_retraction_method`](@extref `ManifoldsBase.default_retraction_method-Tuple{AbstractManifold}`) to a favourite retraction. If this default is set, a `retraction_method=` does not have to be specified.
 * A [`vector_transport_to!`](@extref ManifoldsBase :doc:`vector_transports`)`M, Y, p, X, q)`; it is recommended to set the [`default_vector_transport_method`](@extref `ManifoldsBase.default_vector_transport_method-Tuple{AbstractManifold}`) to a favourite retraction. If this default is set, a `vector_transport_method=` does not have to be specified.
 * A [`copyto!`](@extref `Base.copyto!-Tuple{AbstractManifold, Any, Any}`)`(M, q, p)` and [`copy`](@extref `Base.copy-Tuple{AbstractManifold, Any}`)`(M,p)` for points and similarly `copy(M, p, X)` for tangent vectors.
-* [`get_coordinates!`](@extref `ManifoldsBase.get_coordinates-Tuple{AbstractManifold, Any, Any, ManifoldsBase.AbstractBasis}`)`(M, Y, p, X, b)` and [`get_vector!`](@extref `ManifoldsBase.get_vector-Tuple{AbstractManifold, Any, Any, ManifoldsBase.AbstractBasis}`)`(M, X, p, c, b)` with respect to the [`AbstractBasis`](@extref `ManifoldsBase.AbstractBasis`) `b` provided, which is [`DefaultOrthonormalBasis`](@extref `ManifoldsBase.DefaultOrthonormalBasis`) by default from the `basis=` keyword.
+* [`get_coordinates!`](@extref `ManifoldsBase.get_coordinates`)`(M, Y, p, X, b)` and [`get_vector!`](@extref `ManifoldsBase.get_vector`)`(M, X, p, c, b)` with respect to the [`AbstractBasis`](@extref `ManifoldsBase.AbstractBasis`) `b` provided, which is [`DefaultOrthonormalBasis`](@extref `ManifoldsBase.DefaultOrthonormalBasis`) by default from the `basis=` keyword.
 * An [`is_flat`](@extref `ManifoldsBase.is_flat-Tuple{AbstractManifold}`)`(M)`.
 
 ## Internal helpers

docs/src/solvers/quasi_Newton.md

@@ -129,7 +129,7 @@ The [`quasi_Newton`](@ref) solver requires the following functions of a manifold
 * A [`copyto!`](@extref `Base.copyto!-Tuple{AbstractManifold, Any, Any}`)`(M, q, p)` and [`copy`](@extref `Base.copy-Tuple{AbstractManifold, Any}`)`(M,p)` for points and similarly `copy(M, p, X)` for tangent vectors.
 * By default the tangent vector storing the gradient is initialized calling [`zero_vector`](@extref `ManifoldsBase.zero_vector-Tuple{AbstractManifold, Any}`)`(M,p)`.
 
-Most Hessian approximations further require [`get_coordinates`](@extref `ManifoldsBase.get_coordinates-Tuple{AbstractManifold, Any, Any, ManifoldsBase.AbstractBasis}`)`(M, p, X, b)` with respect to the [`AbstractBasis`](@extref `ManifoldsBase.AbstractBasis`) `b` provided, which is [`DefaultOrthonormalBasis`](@extref `ManifoldsBase.DefaultOrthonormalBasis`) by default from the `basis=` keyword.
+Most Hessian approximations further require [`get_coordinates`](@extref `ManifoldsBase.get_coordinates`)`(M, p, X, b)` with respect to the [`AbstractBasis`](@extref `ManifoldsBase.AbstractBasis`) `b` provided, which is [`DefaultOrthonormalBasis`](@extref `ManifoldsBase.DefaultOrthonormalBasis`) by default from the `basis=` keyword.
 
 
 

tutorials/_quarto.yml

@@ -22,4 +22,4 @@ format:
     variant: -raw_html+tex_math_dollars
     wrap: preserve
 
-jupyter: julia-1.10
+jupyter: julia-1.11