Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add edit dashboards command #1573

Merged
merged 10 commits into from
Dec 1, 2023
Merged
14 changes: 14 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -287,6 +287,20 @@ Use this command to dump objects installed by Fleet as part of a package.

Use this command as an exploratory tool to dump objects as they are installed by Fleet when installing a package. Dumped objects are stored in files as they are returned by APIs of the stack, without any processing.

### `elastic-package edit`

_Context: package_

Use this command to edit assets relevant for the package, e.g. Kibana dashboards.

### `elastic-package edit dashboards`

_Context: package_

Use this command to make dashboards editable.

Pass a comma-separated list of dashboard ids with -d or use the interactive prompt to make managed dashboards editable in Kibana.

### `elastic-package export`

_Context: package_
Expand Down
155 changes: 155 additions & 0 deletions cmd/edit.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,155 @@
// Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
// or more contributor license agreements. Licensed under the Elastic License;
// you may not use this file except in compliance with the Elastic License.

package cmd

import (
"fmt"
"net/url"
"strings"

"github.com/spf13/cobra"

"github.com/elastic/elastic-package/internal/cobraext"
"github.com/elastic/elastic-package/internal/common"
"github.com/elastic/elastic-package/internal/install"
"github.com/elastic/elastic-package/internal/kibana"
"github.com/elastic/elastic-package/internal/stack"
)

const editLongDescription = `Use this command to edit assets relevant for the package, e.g. Kibana dashboards.`

const editDashboardsLongDescription = `Use this command to make dashboards editable.

Pass a comma-separated list of dashboard ids with -d or use the interactive prompt to make managed dashboards editable in Kibana.`

func setupEditCommand() *cobraext.Command {
editDashboardsCmd := &cobra.Command{
Use: "dashboards",
Short: "Make dashboards editable in Kibana",
Long: editDashboardsLongDescription,
Args: cobra.NoArgs,
RunE: editDashboardsCmd,
}
editDashboardsCmd.Flags().StringSliceP(cobraext.DashboardIDsFlagName, "d", nil, cobraext.DashboardIDsFlagDescription)
editDashboardsCmd.Flags().Bool(cobraext.TLSSkipVerifyFlagName, false, cobraext.TLSSkipVerifyFlagDescription)
editDashboardsCmd.Flags().Bool(cobraext.AllowSnapshotFlagName, false, cobraext.AllowSnapshotDescription)

cmd := &cobra.Command{
Use: "edit",
Short: "Edit package assets",
Long: editLongDescription,
}
cmd.AddCommand(editDashboardsCmd)
cmd.PersistentFlags().StringP(cobraext.ProfileFlagName, "p", "", fmt.Sprintf(cobraext.ProfileFlagDescription, install.ProfileNameEnvVar))

return cobraext.NewCommand(cmd, cobraext.ContextPackage)
}

func editDashboardsCmd(cmd *cobra.Command, args []string) error {
cmd.Println("Make Kibana dashboards editable")

dashboardIDs, err := cmd.Flags().GetStringSlice(cobraext.DashboardIDsFlagName)
if err != nil {
return cobraext.FlagParsingError(err, cobraext.DashboardIDsFlagName)
}

common.TrimStringSlice(dashboardIDs)

var opts []kibana.ClientOption
tlsSkipVerify, _ := cmd.Flags().GetBool(cobraext.TLSSkipVerifyFlagName)
if tlsSkipVerify {
opts = append(opts, kibana.TLSSkipVerify())
}

allowSnapshot, _ := cmd.Flags().GetBool(cobraext.AllowSnapshotFlagName)
if err != nil {
return cobraext.FlagParsingError(err, cobraext.AllowSnapshotFlagName)
}

profile, err := cobraext.GetProfileFlag(cmd)
if err != nil {
return err
}

kibanaClient, err := stack.NewKibanaClientFromProfile(profile, opts...)
if err != nil {
return fmt.Errorf("can't create Kibana client: %w", err)
}

jsoriano marked this conversation as resolved.
Show resolved Hide resolved
kibanaVersion, err := kibanaClient.Version()
if err != nil {
return fmt.Errorf("can't get Kibana status information: %w", err)
}

if kibanaVersion.IsSnapshot() {
message := fmt.Sprintf("editing dashboards from a SNAPSHOT version of Kibana (%s) is discouraged. It could lead to invalid dashboards (for example if they use features that are reverted or modified before the final release)", kibanaVersion.Version())
if !allowSnapshot {
return fmt.Errorf("%s. --%s flag can be used to ignore this error", message, cobraext.AllowSnapshotFlagName)
}
fmt.Printf("Warning: %s\n", message)
}

if len(dashboardIDs) == 0 {
dashboardIDs, err = promptDashboardIDs(kibanaClient)
if err != nil {
return fmt.Errorf("prompt for dashboard selection failed: %w", err)
}

if len(dashboardIDs) == 0 {
fmt.Println("No dashboards were found in Kibana.")
return nil
}
}

updatedDashboardIDs := make([]string, 0, len(dashboardIDs))
failedDashboardUpdates := make(map[string]string, len(dashboardIDs))
for _, dashboardID := range dashboardIDs {
err = kibanaClient.SetManagedSavedObject("dashboard", dashboardID, false)
if err != nil {
failedDashboardUpdates[dashboardID] = err.Error()
} else {
updatedDashboardIDs = append(updatedDashboardIDs, dashboardID)
}
}

if len(updatedDashboardIDs) > 0 {
urls, err := dashboardURLs(*kibanaClient, updatedDashboardIDs)
if err != nil {
cmd.Println(fmt.Sprintf("\nFailed to retrieve dashboard URLS: %s", err.Error()))
cmd.Println(fmt.Sprintf("The following dashboards are now editable in Kibana:\n%s", strings.Join(updatedDashboardIDs, "\n")))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will probably never happen, because we were using the URL before. But good to handle the error just in case 👍

} else {
cmd.Println(fmt.Sprintf("\nThe following dashboards are now editable in Kibana:%s", urls))
jillguyonnet marked this conversation as resolved.
Show resolved Hide resolved
}
}

if len(failedDashboardUpdates) > 0 {
errMsgs := make([]string, 0, len(failedDashboardUpdates))
for _, value := range failedDashboardUpdates {
errMsgs = append(errMsgs, value)
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if errors.Join() could help here https://pkg.go.dev/errors#Join. I haven't used it a lot, it was recently added to the standard library.

formattedErrMsgs := strings.Join(errMsgs, "\n")
fmt.Println("")
return fmt.Errorf("failed to make one or more dashboards editable: %s", formattedErrMsgs)
}

fmt.Println("\nDone")
return nil
jsoriano marked this conversation as resolved.
Show resolved Hide resolved
}

func dashboardURLs(kibanaClient kibana.Client, dashboardIDs []string) (string, error) {
kibanaHost := kibanaClient.Address()
kibanaURL, err := url.Parse(kibanaHost)
if err != nil {
return "", fmt.Errorf("failed to retrieve Kibana URL: %w", err)
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am getting this output:

The following dashboards are now editable in Kibana:
https://127.0.0.1:5601#/view/elastic_agent-a148dc70-6b3c-11ed-98de-67bdecd21824
https://127.0.0.1:5601#/view/elastic_agent-1badd650-d136-11ed-b85f-4be0157fc90c

I think we are missing to add the /app/dashboards part here.

Suggested change
}
}
dashboardsURL, err := kibanaURL.JoinPath("app", "dashboards")
if err != nil {
return "", fmt.Errorf("failed to build base dashboards URL: %w", err)
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, sorry, just realised that too, pushed a fix.

var urls strings.Builder
for _, dashboardID := range dashboardIDs {
dashboardURL := *kibanaURL
dashboardURL.Path = "app/dashboards"
dashboardURL.Fragment = "/view/" + dashboardID
fmt.Fprintf(&urls, "\n%s", dashboardURL.String())
}
return urls.String(), nil
}
1 change: 1 addition & 0 deletions cmd/root.go
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ var commands = []*cobraext.Command{
setupCleanCommand(),
setupCreateCommand(),
setupDumpCommand(),
setupEditCommand(),
setupExportCommand(),
setupFormatCommand(),
setupInstallCommand(),
Expand Down
4 changes: 4 additions & 0 deletions docs/howto/create_new_package.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,3 +41,7 @@ _Enter the package directory. For nginx integration, it's: [packages/nginx/](htt
3. Verify the package:
1. Enter the package directory: `cd <new_package>`.
2. Check package correctness: `elastic-package check`.

### Edit package dashboards
jsoriano marked this conversation as resolved.
Show resolved Hide resolved

Once a package has been created, assets may need to be updated, e.g. make use of recently added features. Dashboards can be made editable by using the [`elastic-package edit dashboards` command](https://github.com/elastic/elastic-package/blob/main/docs/howto/make_dashboards_editable.md).
68 changes: 68 additions & 0 deletions docs/howto/make_dashboards_editable.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
# HOWTO: Make dashboards editable in Kibana

## Introduction

As of 8.11, managed assets, including dashboards, are read-only in Kibana. This change was introduced to prevent users from losing changes on package upgrades. Integrations authors, however, need the ability to edit assets in order to adopt new features.

## Making a dashboard editable

Dashboards can be made editable in Kibana by using the `elastic-package edit dashboards` command. This command can eiter be run interactively, allowing manual selection of dashboards, or be passed a comma-separated list of dashboard ids.
jillguyonnet marked this conversation as resolved.
Show resolved Hide resolved

### Using the interactive dashboard selection prompt

Run the following command:
```
elastic-package edit dashboards
```

Use the interactive dashboard selection prompt to select the dashboard(s) that should be made editable.

### Using a comma-separated list of dashboard ids

Pass the list with the `-d` flag:
```
elastic-package edit dashboards -d 123,456,789
```

Each dashboard id will be processed and the outcome of the updates will be listed in the command's final output.

### Command output

The final output will provide the outcome (success or failure) of the update for each dashboard.

For example, assuming the following command:
```
elastic-package edit dashboards -d 123,456,789
```

#### Success

Assuming '123', '456' and '789' are valid dashboard ids and all three updates succeed, the output will be successful and report the URLs of the updated dashboards:
```
Make Kibana dashboards editable

The following dashboards are now editable in Kibana:
https://<kibanaURL>/app/dashboards#/view/123
https://<kibanaURL>/app/dashboards#/view/456
https://<kibanaURL>/app/dashboards#/view/789

Done
```

#### Partial failure

Assuming that `456` is an invalid dashboard id and that the update is successful for ids `123` and `789`, the output will report the URLs of the updated dashboards as well as an error listing the failures:
```
Make Kibana dashboards editable

The following dashboards are now editable in Kibana:
https://<kibanaURL>/app/dashboards#/view/123
https://<kibanaURL>/app/dashboards#/view/789

Error: failed to make one or more dashboards editable: failed to export dashboard 456: could not export saved objects; API status code = 400; response body = {"statusCode":400,"error":"Bad Request","message":"Error fetching objects to export","attributes":{"objects":[{"id":"456","type":"dashboard","error":{"statusCode":404,"error":"Not Found","message":"Saved object [dashboard/456] not found"}}]}}
```

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we have to remember that after modifying the dashboard, it should be exported.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean that this error message could be confusing to the user?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah no, the error is fine, probably I put my comment in the wrong place.
I mean that after following this howto the user is going to be able to edit a dashboard. Maybe we have to mention in some place that after editing the dashboard, the developer should remember to export it with elastic-package export.

### Optional flags

* `allow-snapshot`: to allow exporting dashboards from a Elastic stack SNAPSHOT version
* `tls-skip-verify`: to skip TLS verify
5 changes: 5 additions & 0 deletions internal/kibana/client.go
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,11 @@ func NewClient(opts ...ClientOption) (*Client, error) {
return c, nil
}

// Get client host
func (c *Client) Address() string {
return c.host
}

// Address option sets the host to use to connect to Kibana.
func Address(address string) ClientOption {
return func(c *Client) {
Expand Down