Upgrade Website (#2715)

Summary: Pull Request resolved: #2715 ## Motivation We are already using most of the correct tooling but it has become very out of date and is tied together with scripts that are difficult to maintain. Upgrade our tooling by updating to latest versions and dramatically simplifying our setup/deploy scripts. Includes: - Upgrade Docusaurus from V1 to V3 - Utilize new MDX support in Docusaurus by converting our tutorials to MDX (previously converted to html) - Tutorials link to Colab for easy execution - Website versioning is now handled natively through Docusaurus - New website versions are only created for major and minor releases (not patches) - Build and Deploy workflows have been dramatically simplified - Change Sphinx theme to ReadTheDocs - API Reference is still built using Sphinx but is now hosted by ReadTheDocs. ReadTheDocs also handles versioning. - Legacy website will be frozen and available at [archive.botorch.org](archive.botorch.org) - Served by Vercel from the `archive` branch - Website now supports dark mode | Before | After | | -------- | ------- | | <img width="1280" alt="image" src="https://github.com/user-attachments/assets/5c9f2d8d-6e9b-4712-b818-eca5d558adc7" /> | <img width="1282" alt="image" src="https://github.com/user-attachments/assets/79a247e5-0d84-4202-8d22-9ccaa03af200" /> | | <img width="1278" alt="image" src="https://github.com/user-attachments/assets/a2f2c259-dccf-4c82-a514-dc2bb2b75f16" /> | <img width="1277" alt="image" src="https://github.com/user-attachments/assets/0722fd36-ee4d-478d-b416-ff9c66c2e83a" /> | | <img width="1279" alt="image" src="https://github.com/user-attachments/assets/606529da-2725-4910-b9fc-d060a4713a8f" /> | <img width="1276" alt="image" src="https://github.com/user-attachments/assets/d4358df6-1092-42e3-ae8c-3701772d61a6" /> | | <img width="1277" alt="image" src="https://github.com/user-attachments/assets/aeb54819-c256-49cd-a4ff-2b7ac7c9af15" /> | <img width="1277" alt="image" src="https://github.com/user-attachments/assets/71b34ff9-1aaf-4c39-ade1-7bb346aad317" /> | The commits for most of the major changes are grouped into themed PRs on my fork if that makes the review easier: https://github.com/CristianLara/botorch/pulls?q=is%3Apr+is%3Aclosed ## Considerations - Our tutorials are natively embedded and versioned in the website using Docusaurus, but our sphinx-generated API reference is not. ReadTheDocs will create new version of the sphinx docs for every new release tag in our github repo. - The main branch will now contain copies of the website documentation for every version. This includes the converted tutorial MDX but not the original ipynb files, those are accessible linked to from github using release tags. - Continue using same integrations for Algolia Search and Google Analytics - The number and type of github workflows we have remain the same, but the complexity of the workflows and the supporting scripts has been simplified. Pull Request resolved: #2653 Test Plan: - I've enabled all the workflows in my PR and everything is passing [https://github.com/CristianLara/botorch/actions](https://github.com/CristianLara/botorch/actions) - Pypy deploy and codecov are failing as expected due to missing credentials - Website for my fork is being actively built and served at [cristianlara.me/botorch](cristianlara.me/botorch) - API Reference is being built and served by ReadTheDocs at [https://botorch-dev.readthedocs.io/](https://botorch-dev.readthedocs.io/) Reviewed By: saitcakmak Differential Revision: D67293492 Pulled By: CristianLara fbshipit-source-id: 0d938bd849bd94986cf20587c4420bfba37090d6
pytorch · Jan 31, 2025 · b3f6a38 · b3f6a38
1 parent acae12d
commit b3f6a38
Show file tree

Hide file tree

Showing 86 changed files with 13,090 additions and 9,275 deletions.
diff --git a/.github/workflows/deploy_on_release.yml b/.github/workflows/deploy_on_release.yml
@@ -33,7 +33,7 @@ jobs:
     - name: Set up Python
       uses: actions/setup-python@v5
       with:
-        python-version: "3.10"
+        python-version: "3.12"
     - name: Install dependencies
       run: |
         uv pip install .[test]
@@ -49,11 +49,40 @@ jobs:
       with:
         verbose: true
 
-  publish-versioned-website:
-    name: Publish versioned website
+  check-versions:
     needs: package-deploy-pypi
-    uses: ./.github/workflows/reusable_website.yml
+    name: Check if major or minor version changed
+    runs-on: ubuntu-latest
+    outputs:
+      major_minor_changed: ${{ steps.compare.outputs.major_minor_changed }}
+    steps:
+    - uses: actions/checkout@v4
+      with:
+        fetch-depth: 0
+        fetch-tags: true
+        ref: ${{ github.sha }}
+    - name: Check if major or minor version changed
+      id: compare
+      run: |
+        git fetch --tags --force
+        previous_version=$(git describe --tags --abbrev=0 ${{ github.event.release.tag_name }}^)
+        prev=$(cut -d '.' -f 1-2 <<< $previous_version) # remove patch number
+        prev=${prev#v}                                  # remove optional "v" prefix
+        next=$(cut -d '.' -f 1-2 <<< ${{ github.event.release.tag_name }})
+        next=${next#v}
+
+        echo "Updating from version $previous_version to ${{ github.event.release.tag_name }}"
+        if [[ "$prev" == "$next" ]]; then
+          echo "::warning::Major/Minor version was not changed. Skipping website & docs generation step."
+        else
+          echo major_minor_changed=true >> $GITHUB_OUTPUT
+        fi
+
+  version-and-publish-website:
+    needs: check-versions
+    name: Version and Publish website
+    if: ${{ needs.check-versions.outputs.major_minor_changed == 'true' }}
+    uses: ./.github/workflows/publish_website.yml
     with:
-      publish_versioned_website: true
-      release_tag: ${{ github.event.release.tag_name }}
+      new_version: ${{ github.event.release.tag_name }}
     secrets: inherit
diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml
@@ -25,15 +25,14 @@ jobs:
     - name: Set up Python
       uses: actions/setup-python@v5
       with:
-        python-version: "3.10"
+        python-version: "3.12"
     - name: Install dependencies
       env:
         ALLOW_LATEST_GPYTORCH_LINOP: true
       run: |
         uv pip install git+https://github.com/cornellius-gp/linear_operator.git
         uv pip install git+https://github.com/cornellius-gp/gpytorch.git
-        uv pip install .[dev]
-        uv pip install beautifulsoup4 ipython nbconvert jinja2
+        uv pip install ."[dev, tutorials]"
     - name: Validate Sphinx
       run: |
         python scripts/validate_sphinx.py -p "$(pwd)"
@@ -43,4 +42,4 @@ jobs:
         sphinx-build -WT --keep-going sphinx/source sphinx/build
     - name: Validate and parse tutorials
       run: |
-        python scripts/parse_tutorials.py -w "$(pwd)"
+        python scripts/convert_ipynb_to_mdx.py --clean
diff --git a/.github/workflows/lint.yml b/.github/workflows/lint.yml
@@ -22,7 +22,7 @@ jobs:
       - name: Set up Python
         uses: actions/setup-python@v5
         with:
-          python-version: "3.10"
+          python-version: "3.12"
 
       - name: Install dependencies
         run: uv pip install pre-commit

diff --git a/.github/workflows/nightly.yml b/.github/workflows/nightly.yml
@@ -32,7 +32,7 @@ jobs:
     - name: Set up Python
       uses: actions/setup-python@v5
       with:
-        python-version: "3.10"
+        python-version: "3.12"
     - name: Install dependencies
       env:
         ALLOW_LATEST_GPYTORCH_LINOP: true
@@ -68,9 +68,7 @@ jobs:
   publish-latest-website:
     name: Publish latest website
     needs: [tests-and-coverage-nightly, package-test-deploy-pypi]
-    uses: ./.github/workflows/reusable_website.yml
-    with:
-      publish_versioned_website: false
+    uses: ./.github/workflows/publish_website.yml
     secrets: inherit
 
   run_tutorials:

diff --git a/.github/workflows/publish_website.yml b/.github/workflows/publish_website.yml
@@ -0,0 +1,85 @@
+name: Publish Website
+
+on:
+  workflow_call:
+    inputs:
+      new_version:
+        required: false
+        type: string
+      run_tutorials:
+        required: false
+        type: boolean
+        default: false
+  workflow_dispatch:
+
+
+jobs:
+
+  build-website:
+    runs-on: ubuntu-latest
+    env:
+      # `uv pip ...` requires venv by default. This skips that requirement.
+      UV_SYSTEM_PYTHON: 1
+    steps:
+    - uses: actions/checkout@v4
+      with:
+        ref: 'docusaurus-versions' # release branch
+        fetch-depth: 0
+    - name: Install uv
+      uses: astral-sh/setup-uv@v5
+    - name: Sync release branch with main
+      run: |
+        git config --global user.name "github-actions[bot]"
+        git config --global user.email "41898282+github-actions[bot]@users.noreply.github.com"
+        git merge origin/main
+        # To avoid a large number of commits we don't push this sync commit to github until a new release.
+    - name: Set up Python
+      uses: actions/setup-python@v5
+      with:
+        python-version: "3.12"
+    - if: ${{ !inputs.new_version }}
+      name: Install latest GPyTorch and Linear Operator
+      run: |
+        uv pip install git+https://github.com/cornellius-gp/linear_operator.git
+        uv pip install git+https://github.com/cornellius-gp/gpytorch.git
+    - name: Install dependencies
+      env:
+        ALLOW_LATEST_GPYTORCH_LINOP: true
+        ALLOW_BOTORCH_LATEST: true  # Allow Ax to install w/ new BoTorch release.
+      run: |
+        uv pip install ."[dev, tutorials]"
+        # There may not be a compatible Ax uv pip version, so we use the development version.
+        uv pip install git+https://github.com/facebook/Ax.git
+    - if: ${{ inputs.new_version }}
+      name: Create new docusaurus version
+      run: |
+        python3 scripts/convert_ipynb_to_mdx.py --clean
+        cd website
+        yarn
+        yarn docusaurus docs:version ${{ inputs.new_version }}
+
+        git add --all
+        git commit -m "Create version ${{ inputs.new_version }} of site in Docusaurus"
+        git push --force origin HEAD:docusaurus-versions
+    - name: Build website
+      run: |
+        bash scripts/build_docs.sh -b
+    - name: Upload website build as artifact
+      id: deployment
+      uses: actions/upload-pages-artifact@v3
+      with:
+        path: website/build/
+
+  deploy-website:
+    needs: build-website
+    permissions:
+      pages: write
+      id-token: write
+    environment:
+      name: github-pages
+      url: ${{ steps.deployment.outputs.page_url }}
+    runs-on: ubuntu-latest
+    steps:
+      - name: Deploy to GitHub Pages
+        id: deployment
+        uses: actions/deploy-pages@v4
diff --git a/.github/workflows/reusable_tutorials.yml b/.github/workflows/reusable_tutorials.yml
@@ -41,7 +41,7 @@ jobs:
     - name: Set up Python
       uses: actions/setup-python@v5
       with:
-        python-version: "3.10"
+        python-version: "3.12"
     - name: Fetch all history for all tags and branches
       # We need to do this so setuptools_scm knows how to set the BoTorch version.
       run: git fetch --prune --unshallow

diff --git a/.github/workflows/reusable_website.yml b/.github/workflows/reusable_website.yml
diff --git a/.gitignore b/.gitignore
@@ -94,16 +94,16 @@ sphinx/build/
 website/build/
 website/i18n/
 website/node_modules/
+website/.docusaurus/
 
 ## Generated for tutorials
 website/_tutorials/
 website/static/files/
-website/pages/tutorials/*
-!website/pages/tutorials/index.js
+docs/tutorials/*
+!docs/tutorials/index.mdx
 
 ## Generated for Sphinx
 website/pages/api/
 website/static/js/*
-!website/static/js/mathjax.js
 !website/static/js/code_block_buttons.js
 website/static/_sphinx-sources/
diff --git a/.readthedocs.yaml b/.readthedocs.yaml
@@ -0,0 +1,25 @@
+version: "2"
+
+build:
+  os: "ubuntu-22.04"
+  tools:
+    python: "3.12"
+  jobs:
+    post_install:
+      # Install latest botorch if not on a released version
+      - |
+        tag=$(eval "git name-rev --name-only --tags HEAD")
+        if [ $tag = "undefined" ]; then
+          pip install git+https://github.com/cornellius-gp/linear_operator.git
+          pip install git+https://github.com/cornellius-gp/gpytorch.git
+        fi
+
+python:
+  install:
+    - method: pip
+      path: .
+      extra_requirements:
+        - dev
+
+sphinx:
+  configuration: sphinx/source/conf.py
diff --git a/docs/README.md b/docs/README.md
diff --git a/docs/acquisition.md b/docs/acquisition.md
@@ -36,25 +36,27 @@ functions that consider multiple design points jointly (i.e. $q > 1$).
 An alternative is to use Monte-Carlo (MC) sampling to approximate the integrals.
 An MC approximation of $\alpha$ at $X$ using $N$ MC samples is
 
-$$ \alpha(X) \approx \frac{1}{N} \sum_{i=1}^N a(\xi_{i}) $$
+$$
+\alpha(X) \approx \frac{1}{N} \sum_{i=1}^N a(\xi_{i})
+$$
 
 where $\xi_i \sim \mathbb{P}(f(X) \mid \mathcal{D})$.
 
 For instance, for q-Expected Improvement (qEI), we have:
 
 $$
 \text{qEI}(X) \approx \frac{1}{N} \sum_{i=1}^N \max_{j=1,..., q}
-\bigl\\{ \max(\xi_{ij} - f^\*, 0) \bigr\\},
+\bigl\{ \max(\xi_{ij} - f^*, 0) \bigr\},
 \qquad \xi_{i} \sim \mathbb{P}(f(X) \mid \mathcal{D})
 $$
 
-where $f^\*$ is the best function value observed so far (assuming noiseless
+where $f^*$ is the best function value observed so far (assuming noiseless
 observations). Using the reparameterization trick ([^KingmaWelling2014],
 [^Rezende2014]),
 
 $$
 \text{qEI}(X) \approx \frac{1}{N} \sum_{i=1}^N \max_{j=1,..., q}
-\bigl\\{ \max\bigl( \mu(X)\_j + (L(X) \epsilon_i)\_j - f^\*, 0 \bigr) \bigr\\},
+\bigl\{ \max\bigl( \mu(X)\_j + (L(X) \epsilon_i)\_j - f^*, 0 \bigr) \bigr\},
 \qquad \epsilon_{i} \sim \mathcal{N}(0, I)
 $$
 
@@ -65,10 +67,10 @@ All MC-based acquisition functions in BoTorch are derived from
 [`MCAcquisitionFunction`](../api/acquisition.html#mcacquisitionfunction).
 
 Acquisition functions expect input tensors $X$ of shape
-$\textit{batch_shape} \times q \times d$, where $d$ is the dimension of the
+$\textit{batch\_shape} \times q \times d$, where $d$ is the dimension of the
 feature space, $q$ is the number of points considered jointly, and
-$\textit{batch_shape}$ is the batch-shape of the input tensor. The output
-$\alpha(X)$ will have shape $\textit{batch_shape}$, with each element
+$\textit{batch\_shape}$ is the batch-shape of the input tensor. The output
+$\alpha(X)$ will have shape $\textit{batch\_shape}$, with each element
 corresponding to the respective $q \times d$ batch tensor in the input $X$.
 Note that for analytic acquisition functions, it must be that $q=1$.
 
@@ -135,15 +137,19 @@ summary statistics of the posterior distribution at the evaluated point(s).
 A popular acquisition function is Expected Improvement of a single point
 for a Gaussian posterior, given by
 
-$$ \text{EI}(x) = \mathbb{E}\bigl[
-\max(y - f^\*, 0) \mid y\sim \mathcal{N}(\mu(x), \sigma^2(x))
-\bigr] $$
+$$
+\text{EI}(x) = \mathbb{E}\bigl[
+\max(y - f^*, 0) \mid y\sim \mathcal{N}(\mu(x), \sigma^2(x))
+\bigr]
+$$
 
 where $\mu(x)$ and $\sigma(x)$ are the posterior mean and variance of $f$ at the
-point $x$, and $f^\*$ is again the best function value observed so far (assuming
+point $x$, and $f^*$ is again the best function value observed so far (assuming
 noiseless observations). It can be shown that
 
-$$ \text{EI}(x) = \sigma(x) \bigl( z \Phi(z) + \varphi(z) \bigr)$$
+$$
+\text{EI}(x) = \sigma(x) \bigl( z \Phi(z) + \varphi(z) \bigr)
+$$
 
 where $z = \frac{\mu(x) - f_{\max}}{\sigma(x)}$ and $\Phi$ and $\varphi$ are
 the cdf and pdf of the standard normal distribution, respectively.