Skip to content

Commit fe2db17

Browse files
sri-koundinyansri-koundinyan
authored
Merge branch 'cuda-python-dli' into main
2 parents 7ef3513 + 5fabd2e commit fe2db17

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

49 files changed

+1128778
-0
lines changed

‎tutorials/gpu-cuda-python-tutorial/0.0_Welcome.ipynb‎

Lines changed: 207 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,207 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"id": "11150e25-024f-46cb-8ec5-992864291646",
6+
"metadata": {},
7+
"source": [
8+
"<img src=\"images/cuda-python.jpg\" style=\"float: right;\" />\n",
9+
"\n",
10+
"# CUDA Python Kernel Authoring\n",
11+
"\n",
12+
"_Written by [Katrina Riehl](https://www.linkedin.com/in/katrinariehl)_\n",
13+
"\n",
14+
"**Welcome to the CUDA Python Kernel Authoring tutorial.**\n",
15+
"\n",
16+
"In this tutorial we will cover:\n",
17+
"- What is a GPU and why is it different to a CPU?\n",
18+
"- An overview of the CUDA development model.\n",
19+
"- The CUDA Python ecosystem.\n",
20+
"- Working with NumPy / CuPy style arrays on the GPU.\n",
21+
"- Writing CUDA kernels using cuda.core.\n",
22+
"- Using cuda.cccl.parallel library for algorithmic support.\n",
23+
"\n",
24+
"Attendees will be expected to have a general knowledge of Python and programming concepts, as well as a basic understanding of GPU computing."
25+
]
26+
},
27+
{
28+
"cell_type": "markdown",
29+
"id": "6cee88d4-a78d-4013-be48-d30834a9becd",
30+
"metadata": {},
31+
"source": [
32+
"### Outline\n",
33+
"\n",
34+
"- **0.0** - Overview of CUDA Python ecosystem\n",
35+
"- **1.0** - cuda.core\n",
36+
"- **1.1** - cuda.core exercise solutions\n",
37+
"- **2.0** - cuda.cccl.parallel\n",
38+
"- **2.1** - cuda.cccl.parallel exercise solutions"
39+
]
40+
},
41+
{
42+
"cell_type": "markdown",
43+
"id": "8a1bc4fc-755b-4d79-af13-54787d27a19d",
44+
"metadata": {},
45+
"source": [
46+
"# CPU and GPU Comparison"
47+
]
48+
},
49+
{
50+
"cell_type": "markdown",
51+
"id": "fc72f899-45b3-4951-a63f-f729b21a3d67",
52+
"metadata": {},
53+
"source": [
54+
"The CPU is the most common type of processor for executing your code. CPUs have one or more serial processors which each take single instructions from a stack and **execute them sequentially**.\n",
55+
"\n",
56+
"GPUs are a form of coprocessor which are commonly used for video and image rendering, but are extremely popular in machine learning and data science fields too. GPUs have one or more streaming multiprocessors which take in arrays of instructions and **execute them in parallel**."
57+
]
58+
},
59+
{
60+
"cell_type": "markdown",
61+
"id": "345434c3-0798-4a75-906a-b21670de786c",
62+
"metadata": {},
63+
"source": [
64+
"<figure>\n",
65+
"\n",
66+
"![CPU GPU Comparison](images/cpu-gpu.png)\n",
67+
"\n",
68+
"<figcaption style=\"text-align: center;\"> \n",
69+
" \n",
70+
"Image source <a href=\"https://docs.nvidia.com/cuda/cuda-c-programming-guide/\">https://docs.nvidia.com/cuda/cuda-c-programming-guide/</a>\n",
71+
" \n",
72+
"</figcaption>\n",
73+
"</figure>"
74+
]
75+
},
76+
{
77+
"cell_type": "markdown",
78+
"id": "f1963293-c6ce-4d1c-becc-5861743cd3ec",
79+
"metadata": {},
80+
"source": [
81+
"## What is a kernel?\n",
82+
"\n",
83+
"A kernel is similar to a function, it is a block of code which takes some inputs and is executed by a processor.\n",
84+
"\n",
85+
"The difference between a function and a kernel is:\n",
86+
"- A kernel cannot return anything, it must instead modify memory\n",
87+
"- A kernel must specify its thread hierarchy (threads and blocks)"
88+
]
89+
},
90+
{
91+
"cell_type": "markdown",
92+
"id": "922c26aa-73b0-4436-86db-004e2d8e16bc",
93+
"metadata": {},
94+
"source": [
95+
"## What are grids, threads and blocks (and warps)?\n",
96+
"\n",
97+
"[Threads and blocks](https://en.wikipedia.org/wiki/Thread_block_(CUDA_programming) ) are how you instruct you GPU to process some code in parallel. Our GPU is a parallel processor, so we need to specify how many times we want our kernel to be executed.\n",
98+
"\n",
99+
"Threads have the benefit of having some shared cache memory between them, but there are a limited number of cores on each GPU so we need to break our work down into blocks which will be scheduled and run in parallel on the GPU.\n",
100+
"\n",
101+
"<figure>\n",
102+
"\n",
103+
"![CPU GPU Comparison](images/threads-blocks-warps.png)\n",
104+
"\n",
105+
"<figcaption style=\"text-align: center;\"> \n",
106+
" \n",
107+
"Image source <a href=\"https://docs.nvidia.com/cuda/cuda-c-programming-guide/\">https://docs.nvidia.com/cuda/cuda-c-programming-guide/</a>\n",
108+
" \n",
109+
"</figcaption>\n",
110+
"</figure>\n"
111+
]
112+
},
113+
{
114+
"cell_type": "markdown",
115+
"id": "b688c518-3545-4c0c-ae90-a7aa4bf40690",
116+
"metadata": {},
117+
"source": [
118+
"## So how do you control the GPU?"
119+
]
120+
},
121+
{
122+
"cell_type": "markdown",
123+
"id": "c89b6124-a7cc-4d92-ba6d-43d3bd7cdd55",
124+
"metadata": {},
125+
"source": [
126+
"Executing code on your GPU feels a lot like executing code on a second computer over a network.\n",
127+
"\n",
128+
"If I wanted to send a Python program to another machine to be executed I would need a few things:\n",
129+
"- A way to copy data and code to the remote machine (SCP, SFTP, SMB, NFS, etc)\n",
130+
"- A way to log in and execute programs on that remote machine (SSH, VNC, Remote Desktop, etc)\n",
131+
"\n",
132+
"![CPU GPU Comparison](images/two-computers-network.png)"
133+
]
134+
},
135+
{
136+
"cell_type": "markdown",
137+
"id": "c89e534a-5f6e-4562-a90b-10624dd6afea",
138+
"metadata": {},
139+
"source": [
140+
"To achieve the same things with the GPU we need to use CUDA over PCI. But the idea is still the same &mdash; we need to move data and code to the device and execute that code. \n",
141+
"\n",
142+
"\n",
143+
"\n",
144+
"![CPU GPU Comparison](images/computer-gpu-cuda.png)"
145+
]
146+
},
147+
{
148+
"cell_type": "markdown",
149+
"id": "c8385d5e-5a11-4d30-b574-418f69fbfbf9",
150+
"metadata": {},
151+
"source": [
152+
"## What is CUDA?\n",
153+
"\n",
154+
"[CUDA](https://developer.nvidia.com/cuda-zone) (Compute Unified Device Architecture) is a parallel computing platform and programming model developed by NVIDIA that allows developers to use NVIDIA GPUs for general-purpose processing, not just graphics. It enables programmers to harness the massive parallelism of GPUs to significantly accelerate compute-intensive applications in fields like artificial intelligence, scientific simulations, and data analysis.\n",
155+
"\n",
156+
"## What is CUDA Python?\n",
157+
"\n",
158+
"[CUDA Python](https://nvidia.github.io/cuda-python/latest/) is the home for accessing NVIDIA’s CUDA platform from Python. It consists of multiple components:\n",
159+
"\n",
160+
"- **cuda.core**: Pythonic access to CUDA runtime and other core functionalities\n",
161+
"- **cuda.bindings**: Low-level Python bindings to CUDA C APIs\n",
162+
"- **cuda.pathfinder**: Utilities for locating CUDA components installed in the user’s Python environment\n",
163+
"- **cuda.cccl.cooperative**: A Python module providing CCCL’s reusable block-wide and warp-wide device primitives for use within Numba CUDA kernels\n",
164+
"- **cuda.cccl.parallel**: A Python module for easy access to CCCL’s highly efficient and customizable parallel algorithms, like sort, scan, reduce, transform, etc, that are callable on the host\n",
165+
"- **numba.cuda**: Numba’s target for CUDA GPU programming by directly compiling a restricted subset of Python code into CUDA kernels and device functions following the CUDA execution model.\n",
166+
"- **nvmath-python**: Pythonic access to NVIDIA CPU & GPU Math Libraries, with both host and device (through nvmath.device) APIs. It also provides low-level Python bindings to host C APIs (through nvmath.bindings).\n",
167+
"\n",
168+
"In this tutorial, we will focus on cuda.core and cuda.cccl.parallel."
169+
]
170+
},
171+
{
172+
"cell_type": "code",
173+
"execution_count": null,
174+
"id": "07174e57-4015-4844-97be-c6d6909ccd85",
175+
"metadata": {},
176+
"outputs": [],
177+
"source": []
178+
}
179+
],
180+
"metadata": {
181+
"accelerator": "GPU",
182+
"colab": {
183+
"gpuType": "T4",
184+
"provenance": [],
185+
"toc_visible": true
186+
},
187+
"kernelspec": {
188+
"display_name": "Python 3 (ipykernel)",
189+
"language": "python",
190+
"name": "python3"
191+
},
192+
"language_info": {
193+
"codemirror_mode": {
194+
"name": "ipython",
195+
"version": 3
196+
},
197+
"file_extension": ".py",
198+
"mimetype": "text/x-python",
199+
"name": "python",
200+
"nbconvert_exporter": "python",
201+
"pygments_lexer": "ipython3",
202+
"version": "3.11.7"
203+
}
204+
},
205+
"nbformat": 4,
206+
"nbformat_minor": 5
207+
}

‎tutorials/gpu-cuda-python-tutorial/1.0_CPU_GPU_Comparison.ipynb‎

Lines changed: 151 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,151 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"id": "ff189fc2-802c-42d7-a299-8b8224f0d025",
6+
"metadata": {},
7+
"source": [
8+
"# CPU and GPU Comparison"
9+
]
10+
},
11+
{
12+
"cell_type": "markdown",
13+
"id": "216bd640-6104-45e8-a7c6-09ce9af20f1c",
14+
"metadata": {},
15+
"source": [
16+
"The CPU is the most common type of processor for executing your code. CPUs have one or more serial processors which each take single instructions from a stack and **execute them sequentially**.\n",
17+
"\n",
18+
"GPUs are a form of coprocessor which are commonly used for video and image rendering, but are extremely popular in machine learning and data science fields too. GPUs have one or more streaming multiprocessors which take in arrays of instructions and **execute them in parallel**."
19+
]
20+
},
21+
{
22+
"cell_type": "markdown",
23+
"id": "afdb98ac-89dc-44a6-a3fc-bd5a8374daf7",
24+
"metadata": {},
25+
"source": [
26+
"<figure>\n",
27+
"\n",
28+
"![CPU GPU Comparison](images/cpu-gpu.png)\n",
29+
"\n",
30+
"<figcaption style=\"text-align: center;\"> \n",
31+
" \n",
32+
"Image source <a href=\"https://docs.nvidia.com/cuda/cuda-c-programming-guide/\">https://docs.nvidia.com/cuda/cuda-c-programming-guide/</a>\n",
33+
" \n",
34+
"</figcaption>\n",
35+
"</figure>\n"
36+
]
37+
},
38+
{
39+
"cell_type": "markdown",
40+
"id": "85278019-6617-43dd-9928-9571a683ea6a",
41+
"metadata": {},
42+
"source": [
43+
"## Mythbusters explanation"
44+
]
45+
},
46+
{
47+
"cell_type": "markdown",
48+
"id": "fb6bad2c-63ee-4c28-a42f-9323b1e689d2",
49+
"metadata": {},
50+
"source": [
51+
"This video may help explain the concept visually."
52+
]
53+
},
54+
{
55+
"cell_type": "code",
56+
"execution_count": null,
57+
"id": "85961642-b01c-4ce2-9872-31c3f2186072",
58+
"metadata": {
59+
"tags": []
60+
},
61+
"outputs": [],
62+
"source": [
63+
"from IPython.display import YouTubeVideo\n",
64+
"YouTubeVideo(id='-P28LKWTzrI',width=1000,height=600)"
65+
]
66+
},
67+
{
68+
"cell_type": "markdown",
69+
"id": "fbf0ab63-065c-40e9-9d16-2f51ea192bfc",
70+
"metadata": {},
71+
"source": [
72+
"## So how do you control the GPU?"
73+
]
74+
},
75+
{
76+
"cell_type": "markdown",
77+
"id": "a63ebb5e-3a44-43d6-bab4-22bd98ea523e",
78+
"metadata": {},
79+
"source": [
80+
"Executing code on your GPU feels a lot like executing code on a second computer over a network.\n",
81+
"\n",
82+
"If I wanted to send a Python program to another machine to be executed I would need a few things:\n",
83+
"- A way to copy data and code to the remote machine (SCP, SFTP, SMB, NFS, etc)\n",
84+
"- A way to log in and execute programs on that remote machine (SSH, VNC, Remote Desktop, etc)\n",
85+
"\n",
86+
"![CPU GPU Comparison](images/two-computers-network.png)"
87+
]
88+
},
89+
{
90+
"cell_type": "markdown",
91+
"id": "a302560a-a0bc-4dc9-914c-340817c4b85d",
92+
"metadata": {},
93+
"source": [
94+
"To achieve the same things with the GPU we need to use CUDA over PCI. But the idea is still the same &mdash; we need to move data and code to the device and execute that code. \n",
95+
"\n",
96+
"\n",
97+
"\n",
98+
"![CPU GPU Comparison](images/computer-gpu-cuda.png)"
99+
]
100+
},
101+
{
102+
"cell_type": "markdown",
103+
"id": "888d4144-13bc-487a-a4e1-45995e4b54bc",
104+
"metadata": {},
105+
"source": [
106+
"## What is CUDA?\n",
107+
"\n",
108+
"[CUDA](https://developer.nvidia.com/cuda-zone) is an extension to C++ which allows us to compile GPU code and interact with the GPU.\n",
109+
"\n",
110+
"### But I write Python, not C++!\n",
111+
"\n",
112+
"Over the last few years NVIDIA has invested in bringing CUDA functionality to Python. \n",
113+
"\n",
114+
"Today there are packages like [Numba](https://numba.pydata.org/) which allows us to Just In Time (JIT) compile Python code into something that is compatible with CUDA and provides bindings to transfer data and execute that code.\n",
115+
"\n",
116+
"There are also many high level packages such as [CuPy](https://cupy.dev/), [cuDF](https://github.com/rapidsai/cudf), [cuML](https://github.com/rapidsai/cuml), [cuGraph](https://github.com/rapidsai/cugraph), and more which implement functionality in CUDA C++ and then package that with Python bindings so that it can be used directly from Python. These packages are collectively known as [RAPIDS](https://rapids.ai/).\n",
117+
"\n",
118+
"Lastly, there is also the [CUDA Python](https://developer.nvidia.com/cuda-python) library which provides Cython/Python wrappers for the CUDA driver and runtime APIs.\n",
119+
"\n",
120+
"This tutorial will focus on Numba and RAPIDS."
121+
]
122+
}
123+
],
124+
"metadata": {
125+
"accelerator": "GPU",
126+
"colab": {
127+
"gpuType": "T4",
128+
"provenance": [],
129+
"toc_visible": true
130+
},
131+
"kernelspec": {
132+
"display_name": "Python 3 (ipykernel)",
133+
"language": "python",
134+
"name": "python3"
135+
},
136+
"language_info": {
137+
"codemirror_mode": {
138+
"name": "ipython",
139+
"version": 3
140+
},
141+
"file_extension": ".py",
142+
"mimetype": "text/x-python",
143+
"name": "python",
144+
"nbconvert_exporter": "python",
145+
"pygments_lexer": "ipython3",
146+
"version": "3.11.7"
147+
}
148+
},
149+
"nbformat": 4,
150+
"nbformat_minor": 5
151+
}

0 commit comments

Comments
 (0)