Refactor Cloud ML Examples and integrate with deployment docs

[jacobtomlinson]

TLDR This repository holds loads of really useful example notebooks and documentation on running RAPIDS ML workflows on the cloud but many examples are old or out of date and this repo is hard to maintain. I'd like to refactor this repo and integrate it with the deployment docs to fix this.

#197 (comment)

Challenges with maintaining this repo

• There is no index/toc and maintaining one is hard.
• Cloud deployment instructions get duplicated between notebooks.
  • These instructions go stale and it's hard to update them all.
  • We want https://docs.rapids.ai/deployment/stable/ to be the one source of truth for this.
• Notebooks make opinionated choices about cloud/platform/tool/workflow.
  • We might have a GCP + Kubernetes + Triton FIL + Mortgage example, but not the same for AWS/Azure/etc or using ECS/VMs/etc.
  • We don't want to create a notebook for every option on every axis, there would be too many and it would be hard to keep them up to date.
• Examples are often written with a specific audience or customer in mind and then abandoned once they have served their purpose.

Proposal

I propose that we integrate this repo more tightly with the Deployment documentation repo to try and solve these problems.

• Move as much cloud setup documentation out of notebooks as possible and cross-link to deployment docs pages.
• Use tagging to allow easy navigation between vendor/platform/tool specific reference docs and opinionated workflow examples.
• Build notebooks as a Sphinx site to allow easier navigation and discovery. Either by merging this repo into the deployment one or by setting up some build automation.

Building examples as a website

We could use myst-nb to build the notebooks into markdown files that get synced into the deployment docs repo. They would then get rendered into an examples section of the deployment docs.

The top levels example page could just be a full list of all the notebooks but filterable by tag so that every notebook is discoverable and folks can easily find relevant examples to them.

We could then use those tags to show relevant examples at the bottom of each deployment documentation page.

[jacobtomlinson]

Taking this view of bringing the rapidsai/cloud-ml-examples and rapidsai/deployment repos together into one documentation website also aligns pretty well with Diátaxis.

Diátaxis suggests you should break documentation into four quadrants:

• Tutorials: Lessons that take the reader by the hand through a series of steps to complete a project of some kind. Tutorials are learning-oriented.
• How-to guides: Directions that take the reader through the steps required to solve a real-world problem. How-to guides are goal-oriented.
• Reference guides: Technical descriptions of the machinery and how to operate it. Reference material is information-oriented.
• Explanation: Discussion that clarifies and illuminates a particular topic. Explanation is understanding-oriented.

We pretty much already have this split up clearly with these two projects. I think that consciously thinking about this structure will help us improve content, cross-link and fill gaps better.