[SIP-154] Optional dataset folders

[betodealmeida]

[SIP-154] Optional dataset folders

Motivation

It's not uncommon for core datasets to have hundreds or even thousands (!) of columns/metrics. Exploring these datasets is cumbersome, forcing users to know their names beforehand, so they can search for them in the chart builder UI. Even with small datasets, knowing columns and/or metrics are important can be unclear.

Proposed Change

To address this problem this SIP proposes a worfklow in which metrics and columns in a dataset can be organized into folders (figure 1). The folders are surfaced in the chart builder UI, allowing for quicker and more confident exploration of the metrics and columns (figure 2).

A few clarifications:

• Folders cannot be nested.
• Metrics and columns can only belong to a single folder. If there is a need for having multiple folders with the same metric (say, for different teams), the recommended flow would be to duplicate the dataset and build a different organization in the new dataset.

New or Changed Public Interfaces

1. A new table is needed to store folder name, dataset UUID, and column/metric IDs
2. New feature flag is needed to enable the functionality
3. UI needs to be updated (see screenshots above)
4. Dataset export needs to be updated to include folders, and import to honor them

No new APIs are needed, but the dataset CRUD API will be extended to support folders.

New dependencies

No new dependencies.

Migration Plan and Compatibility

The feature is optional, and old datasets wlil work as today (without any custom folders), so there's no need for migrations.

Rejected Alternatives

An alternative would be to create new datasets with a subset of the metric and columns of a given dataset, but that could lead to the excessive proliferation of datasets, and different datasets could fall out of sync.

[rusackas]

This is a super cool advancement! Let's put it up for [discuss], but you have my +1 when the vote comes!