Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DOCS-1166][DOCS-880][DOCS-1167][DOCS-1168] Automation updates part 1 of 2 #1084

Open
wants to merge 40 commits into
base: main
Choose a base branch
from

Conversation

mdlinville
Copy link
Contributor

@mdlinville mdlinville commented Feb 5, 2025

Update all possible links and references to old Model Registry

Consolidate project-scoped and model registry automations, new Slack notification flow, add redirects

Out of scope:

  • A followup PR will add details about the new API when it is ready.
  • A followup PR will add details about run metrics automations

In technical review

Preview links for major changes:

@mdlinville mdlinville requested a review from a team as a code owner February 5, 2025 23:44
Copy link

cloudflare-workers-and-pages bot commented Feb 5, 2025

Deploying docs with  Cloudflare Pages  Cloudflare Pages

Latest commit: 5e032a0
Status: ✅  Deploy successful!
Preview URL: https://8e5a6719.docodile.pages.dev
Branch Preview URL: https://automation-updates.docodile.pages.dev

View logs

Copy link

@ssisk ssisk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added some comments!

When we do add the docs for run metric events, they need a lot more explanation of how they work than what we have here - we should definitely meet at that time to walk through it

Copy link

@tonyyli-wandb tonyyli-wandb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Partway through but have to step away, so wanted to leave any comments I have so far --

Co-authored-by: Stephen Sisk <ssisk@users.noreply.github.com>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few high level thoughts:

  • For any numbered workflows describing steps in the UI I think it is valuable to have a gif that corresponds to whatever steps we are trying to show
  • The main thing I was hoping to address in this rewrite based on user feedback we got was to reduce the length of the one landing page. It's extremely long and overwhelming to find what you need to find. I think we should structure the automations section into several sub-sections, each a job to be done or a concrete workflow a user might come in trying to understand how to do it from start to finish. Right now it looks like everything is piled into _index which is super long and the user doesn't know what actions (i.e "how to troubleshoot your webhook endpoint") they can find in this doc.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Let's hold off on generating gifs yet. A gif, especially an animated one, suffers from technical debt each time we update the UI, even a minor update,. Also, images are large compared to paragraphs and can make the words seem less legible / can increase the cognitive load of reading the page. First I'd like to make sure that the words are as understandable as possible.
  2. Done! We now have a landing page, an "Event scopes" reference page, a "Create automations" landing page, and specific pages each for Slack and webhook. I left the example payloads and troubleshooting details in the webhook creation page to try to keep the whole task together. I also removed the previous model-registry-automations and project-scoped-automations pages again, which crept back into this PR somehow.

PTAL!

1. Click **Test** to test the webhook. Optionally, provide a payload to test. To refer to a secret the webhook has access to in the payload, prefix its name with `$`. Any payload you specify in this step does not persist, but allows you to test authentication. You configure an automation's payload when you [create the automation]({{< relref "#create-webhook-automation" >}}).

{{% alert %}}
See [Troubleshoot your webhook]({{< relref "#troubleshoot-your-webhook" >}}) to view where the secret and access token are specified in
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think users would find it helpful to see the webhook structure actually in this section as well, as they are in the process of constructing it. Again, I think have a whole section just on creating a webhook, just as you had done in here for secrets would be beneficial.

https://www.notion.so/wandbai/W-B-Post-Request-Structure-1a7e2f5c7ef380a6a446fce56201b00c?pvs=4 - we can include this structure information as well to help users as they are creating it (that doc is just an idea for how to explain the structure, we don't have to use that text exactly).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here is how product board shows the request structure in their developer docs - https://developer.productboard.com/reference/postwebhook-1

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is more relevant to the automation API docs than for the guide about creating webhooks. The new webhook page includes the examples and the troubleshooting content. The API updates are a separate effort that is not in scope for this PR.


### Example webhook payloads

The following tabs demonstrate example payloads based on common use cases. Within the examples they reference the following keys to refer to condition objects in the payload parameters:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a also a metadata template string that is not in this list that we should add

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. Where can I learn more about it?

Now you can [create a webhook automation]({{< relref "#create-webhook-automation" >}}).

#### Create the automation {#create-webhook-automation}
1. Log in to W&B and go to the project page.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm so here is where the nuance of the scope matters. The most common workflow is for users to navigate to a registry and create an automation in there. The instruction defined above is only relevant if you want to create an automation inside a project. And in registry, an automation can either be at the collection or registry level (i.e applies to all collection in a registry) - I think we would want a separate sub-page on the scopes available and how to create an automation for each.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PTAL, I have now structured the Project and Registry tasks into tabs, filled in missing details about both registry collections and project collections, and expanded the Events and Scopes page.

Copy link
Contributor

@noaleetz noaleetz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

first round of feedback from noa

mdlinville and others added 5 commits March 6, 2025 14:08
@mdlinville mdlinville requested a review from noaleetz March 19, 2025 21:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants