Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Doc cleanup for nx-cugraph: fixed typos, cleaned up various descriptions, renamed notebook to match naming convetion. #4478

Merged
merged 11 commits into from
Jul 2, 2024
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions dependencies.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -495,7 +495,9 @@ dependencies:
common:
- output_types: [conda, pyproject]
packages:
- &thrift thriftpy2
# this thriftpy2 entry can be removed entirely (or switched to a '!=')
# once a new release of that project resolves https://github.com/Thriftpy/thriftpy2/issues/281
- &thrift thriftpy2<=0.5.0
python_run_cugraph_service_server:
common:
- output_types: [conda, pyproject]
Expand Down Expand Up @@ -545,9 +547,7 @@ dependencies:
- output_types: [conda]
packages:
- pylibwholegraph==24.8.*
# this thriftpy2 entry can be removed entirely (or switched to a '!=')
# once a new release of that project resolves https://github.com/Thriftpy/thriftpy2/issues/281
- thriftpy2<=0.5.0
- *thrift
test_python_pylibcugraph:
common:
- output_types: [conda, pyproject]
Expand Down
27 changes: 13 additions & 14 deletions docs/cugraph/source/nx_cugraph/nx_cugraph.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,17 @@
### nx_cugraph


Whereas previous versions of cuGraph have included mechanisms to make it
trivial to plug in cuGraph algorithm calls. Beginning with version 24.02, nx-cuGraph
is now a [networkX backend](<https://networkx.org/documentation/stable/reference/utils.html#backends>).
The user now need only [install nx-cugraph](<https://github.com/rapidsai/cugraph/blob/branch-24.08/python/nx-cugraph/README.md#install>)
to experience GPU speedups.
nx-cugraph is a [NetworkX
backend](<https://networkx.org/documentation/stable/reference/utils.html#backends>) that provides GPU acceleration to many popular NetworkX algorithms.

Lets look at some examples of algorithm speedups comparing CPU based NetworkX to dispatched versions run on GPU with nx_cugraph.
By simply [installing and enabling nx-cugraph](<https://github.com/rapidsai/cugraph/blob/HEAD/python/nx-cugraph/README.md#install>), users can see significant speedup on workflows where performance is hindered by the default NetworkX implementation. With nx-cugraph, users can have GPU-based, large-scale performance without changing their familiar and easy-to-use NetworkX code.

Let's look at some examples of algorithm speedups comparing NetworkX with and without GPU acceleration using nx-cugraph.

Each chart has three measurements.
* NX - running the algorithm natively with networkX on CPU.
* nx-cugraph - running with GPU accelerated networkX achieved by simply calling the cugraph backend. This pays the overhead of building the GPU resident object for each algorithm called. This achieves significant improvement but stil isn't compleltely optimum.
* nx-cugraph (preconvert) - This is a bit more complicated since it involves building (precomputing) the GPU resident graph ahead and reusing it for each algorithm.
* NX - default NetworkX, no GPU acceleration
* nx-cugraph - GPU-accelerated NetworkX using nx-cugraph. This involves an internal conversion/transfer of graph data from CPU to GPU memory
* nx-cugraph (preconvert) - GPU-accelerated NetworkX using nx-cugraph with the graph data pre-converted/transferred to GPU


![Ancestors](../images/ancestors.png)
Expand Down Expand Up @@ -44,25 +43,25 @@ user@machine:/# ipython bc_demo.ipy

You will observe a run time of approximately 7 minutes...more or less depending on your cpu.

Run the command again, this time specifiying cugraph as the NetworkX backend of choice.
Run the command again, this time specifying cugraph as the NetworkX backend.
```
user@machine:/# NETWORKX_BACKEND_PRIORITY=cugraph ipython bc_demo.ipy
```
This run will be much faster, typically around 20 seconds depending on your GPU.
```
user@machine:/# NETWORKX_BACKEND_PRIORITY=cugraph ipython bc_demo.ipy
```
There is also an option to add caching. This will dramatically help performance when running multiple algorithms on the same graph.
There is also an option to cache the graph conversion to GPU. This can dramatically improve performance when running multiple algorithms on the same graph.
```
NETWORKX_BACKEND_PRIORITY=cugraph CACHE_CONVERTED_GRAPH=True ipython bc_demo.ipy
NETWORKX_BACKEND_PRIORITY=cugraph NETWORKX_CACHE_CONVERTED_GRAPHS=True ipython bc_demo.ipy
```

When running Python interactively, cugraph backend can be specified as an argument in the algorithm call.
When running Python interactively, the cugraph backend can be specified as an argument in the algorithm call.

For example:
```
nx.betweenness_centrality(cit_patents_graph, k=k, backend="cugraph")
```


The latest list of algorithms that can be dispatched to nx-cuGraph for acceleration is found [here](https://github.com/rapidsai/cugraph/blob/main/python/nx-cugraph/README.md#algorithms).
The latest list of algorithms supported by nx-cugraph can be found [here](https://github.com/rapidsai/cugraph/blob/main/python/nx-cugraph/README.md#algorithms).
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,11 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# Benchmarking Performance of NetworkX with Rapids GPU-based nx_cugraph backend vs on cpu\n",
"# Benchmarking Performance of NetworkX with and without the RAPIDS GPU-based nx-cugraph backend\n",
"# Skip notebook test\n",
"This notebook demonstrates compares the performance of nx_cugraph as a dispatcher for NetworkX algorithms. \n",
"This notebook collects the run times with and without the nx-cugraph backend and graph caching enabled for three popular NetworkX algorithms: Betweenness Centrality, Breadth First Search, and Louvain Community Detection.\n",
"\n",
"We do this by executing Betweenness Centrality, Breadth First Search and Louvain Community Detection, collecting run times with and without nx_cugraph backend and graph caching enabled. nx_cugraph is a registered NetworkX backend. Using it is a zero code change solution.\n",
"\n",
"In the notebook switching to the nx-cugraph backend is done via variables set using the [NetworkX config package](https://networkx.org/documentation/stable/reference/backends.html#networkx.utils.configs.NetworkXConfig) **which requires networkX 3.3 or later !!**\n",
"In this notebook, enabling the nx-cugraph backend will be done using the [NetworkX config API](https://networkx.org/documentation/stable/reference/backends.html#networkx.utils.configs.NetworkXConfig) (which requires NetworkX 3.3 or later)\n",
"\n",
"\n",
"They can be set at the command line as well.\n",
Expand All @@ -19,7 +17,7 @@
"\n",
"\n",
"\n",
"Here is a sample minimal script to demonstrate No-code-change GPU acceleration using nx-cugraph.\n",
"Here is a sample minimal script to demonstrate no-code-change GPU acceleration using nx-cugraph.\n",
"\n",
"----\n",
"bc_demo.ipy:\n",
Expand Down Expand Up @@ -71,7 +69,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"This installs the NetworkX cuGraph dispatcher if not already present."
"This installs nx-cugraph if not already present."
]
},
{
Expand All @@ -92,11 +90,10 @@
"source": [
"This is boiler plate NetworkX code to run:\n",
"* betweenness Centrality\n",
"* Bredth first Search\n",
"* Breadth first Search\n",
"* Louvain community detection\n",
rlratzel marked this conversation as resolved.
Show resolved Hide resolved
"\n",
"and report times. it is completely unaware of cugraph or GPU-based tools.\n",
"[NetworkX configurations](https://networkx.org/documentation/stable/reference/utils.html#backends) can determine how they are run."
"and report times. This code does not require modification to use with nx-cugraph.\n"
]
},
{
Expand All @@ -106,15 +103,15 @@
"outputs": [],
"source": [
"def run_algos(G):\n",
" runtime = time.time()\n",
" starttime = time.time()\n",
" result = nx.betweenness_centrality(G, k=10)\n",
" print (\"Betweenness Centrality time: \" + str(round(time.time() - runtime))+ \" seconds\")\n",
" runtime = time.time()\n",
" print (\"Betweenness Centrality time: \" + str(round(time.time() - starttime))+ \" seconds\")\n",
" starttime = time.time()\n",
" result = nx.bfs_tree(G,source=1)\n",
" print (\"Breadth First Search time: \" + str(round(time.time() - runtime))+ \" seconds\")\n",
" runtime = time.time()\n",
" print (\"Breadth First Search time: \" + str(round(time.time() - starttime))+ \" seconds\")\n",
" starttime = time.time()\n",
" result = nx.community.louvain_communities(G,threshold=1e-04)\n",
" print (\"Louvain time: \" + str(round(time.time() - runtime))+ \" seconds\")\n",
" print (\"Louvain time: \" + str(round(time.time() - starttime))+ \" seconds\")\n",
" return"
]
},
Expand Down Expand Up @@ -146,10 +143,9 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Setting the NetworkX dispatcher with an environment variable or in code using NetworkX config package which is new to [NetworkX 3.3 config](https://networkx.org/documentation/stable/reference/backends.html#networkx.utils.configs.NetworkXConfig).\n",
"Setting the NetworkX dispatcher with an environment variable or in code using the NetworkX config API ([NetworkX 3.3+](https://networkx.org/documentation/stable/reference/backends.html#networkx.utils.configs.NetworkXConfig)).\n",
"\n",
"These convenience settinge allow turning off caching and cugraph dispatching if you want to see how long cpu-only takes.\n",
"This example using an AMD Ryzen Threadripper PRO 3975WX 32-Cores cpu completed in slightly over 40 minutes."
"This example using an AMD Ryzen Threadripper PRO 3975WX 32-Cores CPU completed in slightly over 40 minutes."
]
},
{
Expand All @@ -171,12 +167,12 @@
"if use_cugraph:\n",
" nx.config[\"backend_priority\"]=['cugraph']\n",
"else:\n",
" # Use this setting to turn off the cugraph dispatcher running in legacy cpu mode.\n",
" # Use this setting to use the default NetworkX implementation.\n",
" nx.config[\"backend_priority\"]=[]\n",
"if cache_graph:\n",
" nx.config[\"cache_converted_graphs\"]= True\n",
"else:\n",
" # Use this setting to turn off graph caching which will convertthe NetworkX to a gpu-resident graph each time an algorithm is run.\n",
" # Use this setting to disable caching of graph conversions. This will require nx-cugraph to convert and transfer the native CPU-based graph object to the GPU each time an algorithm is run.\n",
" nx.config[\"cache_converted_graphs\"]= False\n"
]
},
Expand Down
2 changes: 1 addition & 1 deletion python/cugraph-service/client/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ authors = [
license = { text = "Apache 2.0" }
requires-python = ">=3.9"
dependencies = [
"thriftpy2",
"thriftpy2<=0.5.0",
] # This list was generated by `rapids-dependency-file-generator`. To make changes, edit ../../../dependencies.yaml and run `rapids-dependency-file-generator`.
classifiers = [
"Intended Audience :: Developers",
Expand Down
2 changes: 1 addition & 1 deletion python/cugraph-service/server/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ dependencies = [
"numpy>=1.23,<2.0a0",
"rapids-dask-dependency==24.8.*",
"rmm==24.8.*",
"thriftpy2",
"thriftpy2<=0.5.0",
"ucx-py==0.39.*",
] # This list was generated by `rapids-dependency-file-generator`. To make changes, edit ../../../dependencies.yaml and run `rapids-dependency-file-generator`.
classifiers = [
Expand Down
Loading