Skip to content

Commit

Permalink
docs: add instructions for windows wsl
Browse files Browse the repository at this point in the history
  • Loading branch information
l4rm4nd authored Jan 31, 2025
1 parent 078cbea commit 3e9de50
Showing 1 changed file with 90 additions and 0 deletions.
90 changes: 90 additions & 0 deletions examples/ollama-ui/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,3 +21,93 @@ Afterwards, we can browse Open WebUI on `http://127.0.0.1:8080` and register our
> [!TIP]
>
> You likely want to pass a GPU into the Ollama container. Please read [this](https://hub.docker.com/r/ollama/ollama).

## Windows WSL 2 Setup

1. Enable `Nested/Hardware Virtualization` in your BIOS
2. Install Nvidia drivers for Windows
3. Install Docker Desktop for Windows

Then open an administrative PowerShell and proceed as follows:

````
# install wsl
wsl --install
# configure wsl 2 as default
wsl --set-default-version 2
# install ubuntu
wsl --install -d ubuntu
````

After Ubuntu installation, you are prompted for a `username` and `password`.

Afterwards, you will find youself in a Ubuntu terminal.

## Windows Docker Desktop Setup

Open Docker Desktop and enable WSL 2 via the setting `Use WSL 2 based engine`:

![image](https://github.com/user-attachments/assets/85d1270a-80f9-4801-b388-81372af19f43)

Then open `Resources > WSL Integration` and enable Docker for your Ubuntu WSL VM(s):

![image](https://github.com/user-attachments/assets/5b3c7ff2-d865-4e43-b867-cc931516c5d9)

## WSL 2 Ubuntu Setup

After Docker Desktop adjustments, return to the PowerShell with your Ubuntu terminal.

Proceed in Ubuntu terminal with the following commands from [Ollama documentation](https://hub.docker.com/r/ollama/ollama):

````
# add repo
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey \
| sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list \
| sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' \
| sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt-get update
# install nvidia container toolkit
sudo apt-get install -y nvidia-container-toolkit
# add your user to docker group
sudo usermod -aG docker <your-username>
newgrp docker
# test docker and compose
docker -v
docker compose -v
````

> [!WARNING]
> You likely have to restart Docker Desktop and re-open the Ubuntu WSL VM
Now you can make use of Docker in your WSL 2 Ubuntu VM.

1. Store the `docker-compose.yml` from this repo into your WSL 2 Ubuntu VM
2. Start the Docker stack consisting of Ollama and OpenWebUI

````
docker compose up -d
````

Afterwards, you can open a web browser of your choice on your Windows and browse `http://127.0.0.1:8080`.

You will be greeted by OpenWebUI and must setup your administrative user account.

Afterwards:
1. Browse the settings at `http://127.0.0.1:8080/admin/settings`
2. Visit the `Connections` area
3. Disable `OpenAI API`
4. Click the `Manage` download button at Ollama and download your preferred models (see https://ollama.com/library)

![image](https://github.com/user-attachments/assets/a3b03885-3605-439e-9536-82f1933aa5b1)

Unsure which Deepseek R1 model your GPU can run? Read this [Reddit post](https://www.reddit.com/r/LocalLLaMA/comments/1i5wahr/comment/m8wi5tl/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button).

Here my results on Windows 11 with RTX 3080 Ti 12GB VRAM
![image](https://github.com/user-attachments/assets/e8886bfd-65f8-498d-b0e0-db68700352a5)

0 comments on commit 3e9de50

Please sign in to comment.