Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add edit dashboards command #1573

Merged
merged 10 commits into from
Dec 1, 2023
Merged
14 changes: 14 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -287,6 +287,20 @@ Use this command to dump objects installed by Fleet as part of a package.

Use this command as an exploratory tool to dump objects as they are installed by Fleet when installing a package. Dumped objects are stored in files as they are returned by APIs of the stack, without any processing.

### `elastic-package edit`

_Context: package_

Use this command to edit assets relevant for the package, e.g. Kibana dashboards.

### `elastic-package edit dashboards`

_Context: package_

Use this command to make dashboards editable.

Pass a comma-separated list of dashboard ids with -d or use the interactive prompt to make managed dashboards editable in Kibana.

### `elastic-package export`

_Context: package_
Expand Down
155 changes: 155 additions & 0 deletions cmd/edit.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,155 @@
// Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
// or more contributor license agreements. Licensed under the Elastic License;
// you may not use this file except in compliance with the Elastic License.

package cmd

import (
"errors"
"fmt"
"net/url"
"strings"

"github.com/spf13/cobra"

"github.com/elastic/elastic-package/internal/cobraext"
"github.com/elastic/elastic-package/internal/common"
"github.com/elastic/elastic-package/internal/install"
"github.com/elastic/elastic-package/internal/kibana"
"github.com/elastic/elastic-package/internal/stack"
)

const editLongDescription = `Use this command to edit assets relevant for the package, e.g. Kibana dashboards.`

const editDashboardsLongDescription = `Use this command to make dashboards editable.

Pass a comma-separated list of dashboard ids with -d or use the interactive prompt to make managed dashboards editable in Kibana.`

func setupEditCommand() *cobraext.Command {
editDashboardsCmd := &cobra.Command{
Use: "dashboards",
Short: "Make dashboards editable in Kibana",
Long: editDashboardsLongDescription,
Args: cobra.NoArgs,
RunE: editDashboardsCmd,
}
editDashboardsCmd.Flags().StringSliceP(cobraext.DashboardIDsFlagName, "d", nil, cobraext.DashboardIDsFlagDescription)
editDashboardsCmd.Flags().Bool(cobraext.TLSSkipVerifyFlagName, false, cobraext.TLSSkipVerifyFlagDescription)
editDashboardsCmd.Flags().Bool(cobraext.AllowSnapshotFlagName, false, cobraext.AllowSnapshotDescription)

cmd := &cobra.Command{
Use: "edit",
Short: "Edit package assets",
Long: editLongDescription,
}
cmd.AddCommand(editDashboardsCmd)
cmd.PersistentFlags().StringP(cobraext.ProfileFlagName, "p", "", fmt.Sprintf(cobraext.ProfileFlagDescription, install.ProfileNameEnvVar))

return cobraext.NewCommand(cmd, cobraext.ContextPackage)
}

func editDashboardsCmd(cmd *cobra.Command, args []string) error {
cmd.Println("Make Kibana dashboards editable")

dashboardIDs, err := cmd.Flags().GetStringSlice(cobraext.DashboardIDsFlagName)
if err != nil {
return cobraext.FlagParsingError(err, cobraext.DashboardIDsFlagName)
}

common.TrimStringSlice(dashboardIDs)

var opts []kibana.ClientOption
tlsSkipVerify, _ := cmd.Flags().GetBool(cobraext.TLSSkipVerifyFlagName)
if tlsSkipVerify {
opts = append(opts, kibana.TLSSkipVerify())
}

allowSnapshot, _ := cmd.Flags().GetBool(cobraext.AllowSnapshotFlagName)
if err != nil {
return cobraext.FlagParsingError(err, cobraext.AllowSnapshotFlagName)
}

profile, err := cobraext.GetProfileFlag(cmd)
if err != nil {
return err
}

kibanaClient, err := stack.NewKibanaClientFromProfile(profile, opts...)
if err != nil {
return fmt.Errorf("can't create Kibana client: %w", err)
}

jsoriano marked this conversation as resolved.
Show resolved Hide resolved
kibanaVersion, err := kibanaClient.Version()
if err != nil {
return fmt.Errorf("can't get Kibana status information: %w", err)
}

if kibanaVersion.IsSnapshot() {
message := fmt.Sprintf("editing dashboards from a SNAPSHOT version of Kibana (%s) is discouraged. It could lead to invalid dashboards (for example if they use features that are reverted or modified before the final release)", kibanaVersion.Version())
if !allowSnapshot {
return fmt.Errorf("%s. --%s flag can be used to ignore this error", message, cobraext.AllowSnapshotFlagName)
}
fmt.Printf("Warning: %s\n", message)
}

if len(dashboardIDs) == 0 {
dashboardIDs, err = promptDashboardIDs(kibanaClient)
if err != nil {
return fmt.Errorf("prompt for dashboard selection failed: %w", err)
}

if len(dashboardIDs) == 0 {
fmt.Println("No dashboards were found in Kibana.")
return nil
}
}

updatedDashboardIDs := make([]string, 0, len(dashboardIDs))
failedDashboardUpdates := make(map[string]error, len(dashboardIDs))
for _, dashboardID := range dashboardIDs {
err = kibanaClient.SetManagedSavedObject("dashboard", dashboardID, false)
if err != nil {
failedDashboardUpdates[dashboardID] = err
} else {
updatedDashboardIDs = append(updatedDashboardIDs, dashboardID)
}
}

if len(updatedDashboardIDs) > 0 {
urls, err := dashboardURLs(*kibanaClient, updatedDashboardIDs)
if err != nil {
cmd.Println(fmt.Sprintf("\nFailed to retrieve dashboard URLS: %s", err.Error()))
cmd.Println(fmt.Sprintf("The following dashboards are now editable in Kibana:\n%s", strings.Join(updatedDashboardIDs, "\n")))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will probably never happen, because we were using the URL before. But good to handle the error just in case 👍

} else {
cmd.Println(fmt.Sprintf("\nThe following dashboards are now editable in Kibana:%s\n\nRemember to export modified dashboards with elastic-package export dashboards", urls))
}
}

if len(failedDashboardUpdates) > 0 {
var combinedErr error
for _, err := range failedDashboardUpdates {
combinedErr = errors.Join(combinedErr, err)
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if errors.Join() could help here https://pkg.go.dev/errors#Join. I haven't used it a lot, it was recently added to the standard library.

fmt.Println("")
return fmt.Errorf("failed to make one or more dashboards editable: %s", combinedErr.Error())
}

fmt.Println("\nDone")
return nil
jsoriano marked this conversation as resolved.
Show resolved Hide resolved
}

func dashboardURLs(kibanaClient kibana.Client, dashboardIDs []string) (string, error) {
kibanaHost := kibanaClient.Address()
kibanaURL, err := url.Parse(kibanaHost)
if err != nil {
return "", fmt.Errorf("failed to retrieve Kibana URL: %w", err)
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am getting this output:

The following dashboards are now editable in Kibana:
https://127.0.0.1:5601#/view/elastic_agent-a148dc70-6b3c-11ed-98de-67bdecd21824
https://127.0.0.1:5601#/view/elastic_agent-1badd650-d136-11ed-b85f-4be0157fc90c

I think we are missing to add the /app/dashboards part here.

Suggested change
}
}
dashboardsURL, err := kibanaURL.JoinPath("app", "dashboards")
if err != nil {
return "", fmt.Errorf("failed to build base dashboards URL: %w", err)
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, sorry, just realised that too, pushed a fix.

var urls strings.Builder
for _, dashboardID := range dashboardIDs {
dashboardURL := *kibanaURL
dashboardURL.Path = "app/dashboards"
dashboardURL.Fragment = "/view/" + dashboardID
fmt.Fprintf(&urls, "\n%s", dashboardURL.String())
}
return urls.String(), nil
}
1 change: 1 addition & 0 deletions cmd/root.go
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ var commands = []*cobraext.Command{
setupCleanCommand(),
setupCreateCommand(),
setupDumpCommand(),
setupEditCommand(),
setupExportCommand(),
setupFormatCommand(),
setupInstallCommand(),
Expand Down
28 changes: 22 additions & 6 deletions docs/howto/create_new_package.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,13 @@ The `elastic-package` tool can be used to bootstrap a new package or add a data
It's advised to use `elastic-package create` to build new package rather than copying sources of an existing package.
This will ensure that you're following latest recommendations for the package format.

### Create new package
## Create new package

#### Prerequisites
### Prerequisites

_Pick the directory where you'd like to create a new package. For integrations, it's: [packages/](https://github.com/elastic/integrations/tree/master/packages)._

#### Steps
### Steps

1. Bootstrap new package using the TUI wizard: `elastic-package create package`.
2. Adjust the created package manually:
Expand All @@ -24,13 +24,13 @@ _Pick the directory where you'd like to create a new package. For integrations,
1. Enter the package directory: `cd <new_package>`.
2. Check package correctness: `elastic-package check`.

### Add data stream
## Add data stream

#### Prerequisites
### Prerequisites

_Enter the package directory. For nginx integration, it's: [packages/nginx/](https://github.com/elastic/integrations/tree/master/packages/nginx)._

#### Steps
### Steps

1. Bootstrap new data stream using the TUI wizard: `elastic-package create data-stream`.
2. Adjust the created data stream manually:
Expand All @@ -41,3 +41,19 @@ _Enter the package directory. For nginx integration, it's: [packages/nginx/](htt
3. Verify the package:
1. Enter the package directory: `cd <new_package>`.
2. Check package correctness: `elastic-package check`.

## Export package dashboards

Once the package assets are defined, these should be exported using the `elastic-package export` command. As dashboards are the only type of exportable asset, the command is:
```
elastic-package export dashboards
```

Used this way, this command will open an interactive prompt for dashboard selection. Alternatively, a comma-separated list of dashboard ids can be provided with the `-d` flag:
```
elastic-package export dashboards -d 123,345,789
```

### Edit package dashboards
jsoriano marked this conversation as resolved.
Show resolved Hide resolved

When updating an existing package, assets may need to be updated, e.g. to make use of recently added features. Dashboards can be made editable by using the [`elastic-package edit dashboards` command](https://github.com/elastic/elastic-package/blob/main/docs/howto/make_dashboards_editable.md).
70 changes: 70 additions & 0 deletions docs/howto/make_dashboards_editable.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
# HOWTO: Make dashboards editable in Kibana

## Introduction

As of 8.11, managed assets, including dashboards, are read-only in Kibana. This change was introduced to prevent users from losing changes on package upgrades. Integrations authors, however, need the ability to edit assets in order to adopt new features.

## Making a dashboard editable

Dashboards can be made editable in Kibana by using the `elastic-package edit dashboards` command. This command can either be run interactively, allowing manual selection of dashboards, or be passed a comma-separated list of dashboard ids.

NB: after modifying dashboards, these need to be exported using `elastic-package export dashboards`.

### Using the interactive dashboard selection prompt

Run the following command:
```
elastic-package edit dashboards
```

Use the interactive dashboard selection prompt to select the dashboard(s) that should be made editable.

### Using a comma-separated list of dashboard ids

Pass the list with the `-d` flag:
```
elastic-package edit dashboards -d 123,456,789
```

Each dashboard id will be processed and the outcome of the updates will be listed in the command's final output.

### Command output

The final output will provide the outcome (success or failure) of the update for each dashboard.

For example, assuming the following command:
```
elastic-package edit dashboards -d 123,456,789
```

#### Success

Assuming '123', '456' and '789' are valid dashboard ids and all three updates succeed, the output will be successful and report the URLs of the updated dashboards:
```
Make Kibana dashboards editable

The following dashboards are now editable in Kibana:
https://<kibanaURL>/app/dashboards#/view/123
https://<kibanaURL>/app/dashboards#/view/456
https://<kibanaURL>/app/dashboards#/view/789

Done
```

#### Partial failure

Assuming that `456` is an invalid dashboard id and that the update is successful for ids `123` and `789`, the output will report the URLs of the updated dashboards as well as an error listing the failures:
```
Make Kibana dashboards editable

The following dashboards are now editable in Kibana:
https://<kibanaURL>/app/dashboards#/view/123
https://<kibanaURL>/app/dashboards#/view/789

Error: failed to make one or more dashboards editable: failed to export dashboard 456: could not export saved objects; API status code = 400; response body = {"statusCode":400,"error":"Bad Request","message":"Error fetching objects to export","attributes":{"objects":[{"id":"456","type":"dashboard","error":{"statusCode":404,"error":"Not Found","message":"Saved object [dashboard/456] not found"}}]}}
```

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we have to remember that after modifying the dashboard, it should be exported.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean that this error message could be confusing to the user?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah no, the error is fine, probably I put my comment in the wrong place.
I mean that after following this howto the user is going to be able to edit a dashboard. Maybe we have to mention in some place that after editing the dashboard, the developer should remember to export it with elastic-package export.

### Optional flags

* `allow-snapshot`: to allow exporting dashboards from a Elastic stack SNAPSHOT version
* `tls-skip-verify`: to skip TLS verify
5 changes: 5 additions & 0 deletions internal/kibana/client.go
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,11 @@ func NewClient(opts ...ClientOption) (*Client, error) {
return c, nil
}

// Get client host
func (c *Client) Address() string {
return c.host
}

// Address option sets the host to use to connect to Kibana.
func Address(address string) ClientOption {
return func(c *Client) {
Expand Down