Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add IBM Cloud IKS docs #49

Draft
wants to merge 32 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
bd050ee
Create single-node
marifse Sep 6, 2022
2cc0942
Update single-node
marifse Sep 6, 2022
b0e48a7
Rename single-node to single-node.md
marifse Sep 6, 2022
91f8358
Update single-node.md
marifse Sep 6, 2022
68ece07
Update single-node.md
marifse Sep 6, 2022
bb7e96d
Update single-node.md
marifse Sep 6, 2022
a263762
Create index.md
marifse Sep 6, 2022
f4f36d4
Update index.md
marifse Sep 6, 2022
e209dbe
Update index.md
marifse Sep 6, 2022
ac4358e
Create iks.md
marifse Sep 6, 2022
ca89675
Update iks.md
marifse Sep 6, 2022
ea4537a
Update iks.md
marifse Sep 6, 2022
387530d
Update iks.md
marifse Sep 6, 2022
89b342e
Update iks.md
marifse Sep 7, 2022
2227523
Update iks.md
marifse Sep 7, 2022
99582b1
Update iks.md
marifse Sep 7, 2022
041333f
Update iks.md
marifse Sep 7, 2022
d19704d
Update iks.md
marifse Sep 7, 2022
225d05f
Update iks.md
marifse Sep 7, 2022
fb4a85b
Update iks.md
marifse Sep 7, 2022
aea3f04
Update iks.md
marifse Sep 7, 2022
0660489
Update iks.md
marifse Sep 7, 2022
26066a1
Update iks.md
marifse Sep 7, 2022
0383716
Delete source/cloud/IBM directory
marifse Sep 7, 2022
c21aede
Create index.md
marifse Sep 7, 2022
e3dc7b6
Create single-node.md
marifse Sep 7, 2022
656cd0a
Create iks.md
marifse Sep 7, 2022
fdda35d
Update iks.md
marifse Sep 7, 2022
4848cd7
Fix linting
jacobtomlinson Oct 19, 2022
0a39e34
Formatting, layout and reuse of existing documentation
jacobtomlinson Oct 19, 2022
24f06cd
Merge branch 'main' of https://github.com/rapidsai/deployment into ma…
jacobtomlinson Oct 19, 2022
cc9268a
Merge branch 'main' of https://github.com/rapidsai/deployment into iks
jacobtomlinson Oct 20, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
76 changes: 76 additions & 0 deletions source/cloud/ibm/iks.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
# IBM Kubernetes Service (IKS)

RAPIDS can be deployed on IBM Cloud via IBM Cloud managed Kubernetes service (IKS) using any of the [supported Kubernetes installation methods](../../platforms/kubernetes).

## Install pre-requisites

Install and configure dependencies in your local environment: [kubectl](https://kubernetes.io/docs/tasks/tools/), [helm](https://helm.sh/), [IBM cloud cli](https://cloud.ibm.com/docs/cli?topic=cli-getting-started) and [IBM Kubernetes Service (KS) plugin](https://cloud.ibm.com/docs/containers?topic=containers-cs_cli_install).

## Login to IBM CLI

```shell
$ ibmcloud login -a cloud.ibm.com -r <region>
$ ibmcloud target -g <resource group>
```

```{note}
You can list regions with `$ ibmcloud regions` and resource groups with `$ ibmcloud resource groups`.
```

## Create a Kubernetes cluster

```shell
$ ibmcloud ks cluster create classic \
--name <CLUSTER_NAME> \
--zone dal10 \
--flavor gx2-8x64x1v100 \
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am unable to find any gx2 flavor instances, are these still available? Do we need to enable something specific on our account.

cc @marifse

$ ibmcloud ks flavors --zone dal10
OK
For more information about these flavors, see 'https://ibm.biz/flavors'
Name                      Cores   Memory   Network Speed   OS             Server Type   Storage   Secondary Storage   Flavor Class   Provider
b3c.16x64                 16      64GB     1000Mbps        UBUNTU_18_64   virtual       25GB      100GB               -              classic
b3c.16x64.300gb           16      64GB     1000Mbps        UBUNTU_18_64   virtual       25GB      300GB               -              classic
b3c.32x128                32      128GB    1000Mbps        UBUNTU_18_64   virtual       25GB      100GB               -              classic
b3c.4x16                  4       16GB     1000Mbps        UBUNTU_18_64   virtual       25GB      100GB               -              classic
b3c.56x242                56      242GB    1000Mbps        UBUNTU_18_64   virtual       25GB      100GB               -              classic
b3c.8x32                  8       32GB     1000Mbps        UBUNTU_18_64   virtual       25GB      100GB               -              classic
c3c.16x16                 16      16GB     1000Mbps        UBUNTU_18_64   virtual       25GB      100GB               -              classic
c3c.16x32                 16      32GB     1000Mbps        UBUNTU_18_64   virtual       25GB      100GB               -              classic
c3c.32x32                 32      32GB     1000Mbps        UBUNTU_18_64   virtual       25GB      100GB               -              classic
c3c.32x64                 32      64GB     1000Mbps        UBUNTU_18_64   virtual       25GB      100GB               -              classic
m3c.16x128                16      128GB    1000Mbps        UBUNTU_18_64   virtual       25GB      100GB               -              classic
m3c.30x240                30      240GB    1000Mbps        UBUNTU_18_64   virtual       25GB      100GB               -              classic
m3c.48x384                48      384GB    1000Mbps        UBUNTU_18_64   virtual       25GB      100GB               -              classic
m3c.4x32                  4       32GB     1000Mbps        UBUNTU_18_64   virtual       25GB      100GB               -              classic
m3c.56x448                56      448GB    1000Mbps        UBUNTU_18_64   virtual       25GB      100GB               -              classic
m3c.64x512                64      512GB    1000Mbps        UBUNTU_18_64   virtual       25GB      100GB               -              classic
m3c.8x64                  8       64GB     1000Mbps        UBUNTU_18_64   virtual       25GB      100GB               -              classic
mb4c.20x192               20      192GB    10000Mbps       UBUNTU_18_64   physical      2TB       960GB               -              classic
mb4c.20x384               20      384GB    10000Mbps       UBUNTU_18_64   physical      2TB       960GB               -              classic
mb4c.20x64                20      64GB     10000Mbps       UBUNTU_18_64   physical      2TB       960GB               -              classic
mb4c.20x64.2x1.9tb.ssd    20      64GB     10000Mbps       UBUNTU_18_64   physical      2TB       960GB               -              classic
mb4c.32x384.3.8tb.ssd     32      384GB    10000Mbps       UBUNTU_18_64   physical      2TB       1.92TB              -              classic
mb4c.32x384.6x3.8tb.ssd   32      384GB    10000Mbps       UBUNTU_18_64   physical      2TB       1.92TB              -              classic
mb4c.32x768.3.8tb.ssd     32      768GB    10000Mbps       UBUNTU_18_64   physical      2TB       960GB               -              classic
mb4c.48x1536              48      1536GB   10000Mbps       UBUNTU_18_64   physical      2TB       960GB               -              classic
me4c.4x32                 4       32GB     10000Mbps       UBUNTU_18_64   physical      2TB       2TB                 -              classic
me4c.4x32.1.9tb.ssd       4       32GB     10000Mbps       UBUNTU_18_64   physical      2TB       960GB               -              classic
mg4c.32x384.2xp100        32      384GB    10000Mbps       UBUNTU_18_64   physical      2TB       960GB               -              classic
mg4c.48x384.2xv100        48      384GB    10000Mbps       UBUNTU_18_64   physical      2TB       960GB               -              classic
u3c.2x4†                  2       4GB      1000Mbps        UBUNTU_18_64   virtual       25GB      100GB               -              classic

Copy link
Contributor

@marifse marifse Oct 20, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hey @jacobtomlinson! thank you four response. To provision IBM Kubernetes cluster, you can use the GPU enabled mg4c.32x384.2xp100 (bare-metal server) instance, though it will incur charges per month one time not hourly, so would be careful. And for your information the gx2-8x64x1v100 is only available for Virtual Server Instance, which is a virtual instance and charged per hour basis. Thank you

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the response @marifse.

I'm apprehensive about providing instructions on deploying a very expensive cluster that bills monthly 😅. Your original instructions used gx2-8x64x1v100 which I've just copied over here, is there any way to get per-hour billing for GPUs on IKS?

We also need to test these instructions for every RAPIDS release (every two months) and right now this means launching two nodes and checking that everything runs. We can't really justify ~$1k per test because of a limitation around the billing period.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@marifse gentle nudge here, do you have any suggestions on how we can safely document IKS?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jacobtomlinson sorry for being late, i have communicated to IBM guy, for now there is no option in IKS (IBM Kubernetes Services) to use GPU enabled instance without bare-metal profile.
For testing the instructions for every RAPIDS release, i have communicated to IBM guy, he will sort out some solution for this, as soon as he will be back to work, as he is on leave for next 15 days.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @marifse I'll leave this as a draft in the meantime, but the single-node docs are already merged and will be in the next stable release which we hope to get out this week.

--hardware dedicated \
--workers 1 \
--version <kubernetes_version>
```

`<CLUSTER_NAME>` = Name of the IKS cluster. This will be auto generated if not specified. <br>
`<kubernetes_version>` = Kubernetes version, the tested version for this deployment is 1.21.14. <br>

Upon successful creation, you would get the cluster id, note that down, it will be required in the next step to connect to the cluster.

## Connect to the cluster

```shell
$ ibmcloud ks cluster config --cluster <cluster_id>
```

`<cluster_id>` = When creating the cluster using IBM KS CLI, use that cluster id to connect to the cluster.

## Install GPU drivers

```shell
$ helm repo add nvidia https://helm.ngc.nvidia.com/nvidia
$ helm repo update
$ helm install --wait --generate-name \
-n gpu-operator --create-namespace \
nvidia/gpu-operator
```

## Install RAPIDS

Follow any of the [Kubernetes installation methods to install and use RAPIDS](../../platforms/kubernetes).

## Delete the cluster

When you are finished delete the Kubernetes cluster.

Before you delete the cluster you need to manually delete services running in the cluster with external IPs to release network resources.

```shell
$ kubectl get svc --all-namespaces
$ kubectl delete svc <SERVICE_NAME>
```

`<SERVICE_NAME>` = Name of the services which have an `EXTERNAL-IP` value.

Delete the cluster and its associated nodes.

```shell
$ ibmcloud ks cluster rm --cluster <cluster_name_or_ID>
```
1 change: 1 addition & 0 deletions source/cloud/ibm/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ maxdepth: 2
caption: IBM Cloud
---
single-node
iks
```

RAPIDS can be deployed on IBM Cloud in several ways. See the
Expand Down