Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use D: for Windows installations on Azure #2076

Merged
merged 12 commits into from
Oct 15, 2024
4 changes: 2 additions & 2 deletions conda_smithy/data/conda-forge.json
Original file line number Diff line number Diff line change
Expand Up @@ -131,7 +131,7 @@
"$ref": "#/$defs/AzureRunnerSettings"
}
],
"description": "Windows-specific settings for runners"
"description": "Windows-specific settings for runners. Some important variables you can set are:\n\n- `CONDA_BLD_PATH`: Location of the conda-build workspace. Defaults to `D:\\bld`\n- `MINIFORGE_HOME`: Location of the base environment installation. Defaults to\n `D:\\Miniforge`.\n- `SET_PAGEFILE`: `\"True\"` to increase the pagefile size via conda-forge-ci-setup.\n\nIf you are running out of space in `D:`, consider changing to `C:`.\nIt's a slower drive but has more space available. We recommend you keep\nboth `CONDA_BLD_PATH` and `MINIFORGE_HOME` in the same drive for performance."
},
"user_or_org": {
"anyOf": [
Expand Down Expand Up @@ -2265,7 +2265,7 @@
"type": "null"
}
],
"description": "Azure Pipelines CI settings. This is usually read-only and should not\nnormally be manually modified. Tools like conda-smithy may modify this, as needed.\nFor example:\n\n```yaml\nazure:\n # flag for forcing the building all supported providers\n force: False\n # toggle for storing the conda build_artifacts directory (including the\n # built packages) as an Azure pipeline artifact that can be downloaded\n store_build_artifacts: False\n # toggle for freeing up some extra space on the default Azure Pipelines\n # linux image before running the Docker container for building\n free_disk_space: False\n # limit the amount of CI jobs running concurrently at a given time\n # each OS will get its proportional share of the configured value\n max_parallel: 25\n```\n\nBelow is an example configuration for setting up a self-hosted Azure agent for Linux:\n\n```yaml\nazure:\n settings_linux:\n pool:\n name: your_local_pool_name\n demands:\n - some_key -equals some_value\n workspace:\n clean: all\n strategy:\n maxParallel: 1\n```\n\nBelow is an example configuration for adding a swapfile on an Azure agent for Linux:\n\n```yaml\nazure:\n settings_linux:\n swapfile_size: 10GiB\n```"
"description": "Azure Pipelines CI settings. This is usually read-only and should not\nnormally be manually modified. Tools like conda-smithy may modify this, as needed.\nFor example:\n\n```yaml\nazure:\n # flag for forcing the building all supported providers\n force: False\n # toggle for storing the conda build_artifacts directory (including the\n # built packages) as an Azure pipeline artifact that can be downloaded\n store_build_artifacts: False\n # toggle for freeing up some extra space on the default Azure Pipelines\n # linux image before running the Docker container for building\n free_disk_space: False\n # limit the amount of CI jobs running concurrently at a given time\n # each OS will get its proportional share of the configured value\n max_parallel: 25\n```\n\nBelow is an example configuration for setting up a self-hosted Azure agent for Linux:\n\n```yaml\nazure:\n settings_linux:\n pool:\n name: your_local_pool_name\n demands:\n - some_key -equals some_value\n workspace:\n clean: all\n strategy:\n maxParallel: 1\n```\n\nBelow is an example configuration for adding a swapfile on an Azure agent for Linux:\n\n```yaml\nazure:\n settings_linux:\n swapfile_size: 10GiB\n```\n\nIf you need more space on Windows, you can use `C:` at the cost of IO performance:\n\n```yaml\nazure:\n settings_win:\n variables:\n CONDA_BLD_PATH: \"C:\\bld\"\n MINIFORGE_HOME: \"C:\\Miniforge\"\n```"
},
"drone": {
"anyOf": [
Expand Down
1 change: 1 addition & 0 deletions conda_smithy/data/conda-forge.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ azure:
timeoutInMinutes: 360
variables:
CONDA_BLD_PATH: D:\\bld\\
MINIFORGE_HOME: D:\Miniforge
UPLOAD_TEMP: D:\\tmp
store_build_artifacts: false
timeout_minutes: null
Expand Down
42 changes: 33 additions & 9 deletions conda_smithy/schema.py
Original file line number Diff line number Diff line change
Expand Up @@ -211,11 +211,25 @@ class AzureConfig(BaseModel):
default_factory=lambda: AzureRunnerSettings(
pool={"vmImage": "windows-2022"},
variables={
"MINIFORGE_HOME": "D:\\Miniforge",
"CONDA_BLD_PATH": "D:\\\\bld\\\\",
"UPLOAD_TEMP": "D:\\\\tmp",
},
),
description="Windows-specific settings for runners",
description=cleandoc(
"""
Windows-specific settings for runners. Some important variables you can set are:

- `CONDA_BLD_PATH`: Location of the conda-build workspace. Defaults to `D:\\bld`
- `MINIFORGE_HOME`: Location of the base environment installation. Defaults to
`D:\\Miniforge`.
- `SET_PAGEFILE`: `"True"` to increase the pagefile size via conda-forge-ci-setup.

If you are running out of space in `D:`, consider changing to `C:`.
It's a slower drive but has more space available. We recommend you keep
both `CONDA_BLD_PATH` and `MINIFORGE_HOME` in the same drive for performance.
"""
),
)

user_or_org: Optional[Union[str, Nullable]] = Field(
Expand Down Expand Up @@ -1237,14 +1251,14 @@ class ConfigModel(BaseModel):
```yaml
azure:
settings_linux:
pool:
name: your_local_pool_name
demands:
- some_key -equals some_value
workspace:
clean: all
strategy:
maxParallel: 1
pool:
name: your_local_pool_name
demands:
- some_key -equals some_value
workspace:
clean: all
strategy:
maxParallel: 1
```

Below is an example configuration for adding a swapfile on an Azure agent for Linux:
Expand All @@ -1254,6 +1268,16 @@ class ConfigModel(BaseModel):
settings_linux:
swapfile_size: 10GiB
```

If you need more space on Windows, you can use `C:` at the cost of IO performance:

```yaml
azure:
settings_win:
variables:
CONDA_BLD_PATH: "C:\\bld"
MINIFORGE_HOME: "C:\\Miniforge"
```
"""
),
)
Expand Down
4 changes: 2 additions & 2 deletions conda_smithy/templates/azure-pipelines-win.yml.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -27,10 +27,10 @@ jobs:
urllib.request.urlretrieve(url, path)

- script: |
start /wait "" %BUILD_ARTIFACTSTAGINGDIRECTORY%\Miniforge.exe /InstallationType=JustMe /RegisterPython=0 /S /D=C:\Miniforge
start /wait "" %BUILD_ARTIFACTSTAGINGDIRECTORY%\Miniforge.exe /InstallationType=JustMe /RegisterPython=0 /S /D=$(MINIFORGE_HOME)
displayName: Install Miniforge

- powershell: Write-Host "##vso[task.prependpath]C:\Miniforge\Scripts"
- powershell: Write-Host "##vso[task.prependpath]$(MINIFORGE_HOME)\Scripts"
displayName: Add conda to PATH

- script: |
Expand Down
23 changes: 23 additions & 0 deletions news/2076-install-to-D.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
**Added:**

* <news item>

**Changed:**

* Use the faster ``D:\`` drive for Miniforge installations on Windows and expose this path as the ``MINIFORGE_HOME`` variable. (#2076)

**Deprecated:**

* <news item>

**Removed:**

* <news item>

**Fixed:**

* <news item>

**Security:**

* <news item>
Loading