tags api v2

[brharrington]

Still need to see exactly what form this takes and whether it can be done as an add-ons to v1 without breaking compatibility. Should try to address:

Time based restrictions

Internally the tags api is only applied to the 6h tier. This is done for several reasons:

• UI heavily depends on inefficient operations that do not scale well to historical clusters with more data per node.
• The transient nature of many resources means garbage accumulates quickly and users want it out of the api. This was a frequent complaint in Epic, the predecessor to Atlas.

However, the time based restriction causes a number of issues with some batch type operations that may not have run in the last 6h. For example canary analysis jobs.

Should the tags api try to support start and end parameters like the graph api? How much overhead would this add to the index?

Pagination

Pagination is required because we wanted to avoid users just downloading everything in particular with large sets where there are millions of entries. A number of suggestions that have come up:

• Use standard link headers for the pagination. Currently there is a custom header for some legacy reasons.
• Allow some form of auto-pagination. This is doable, but need to make sure it doesn't cause any issues for other users.

Multiple Results

There have been some suggestions it would be helpful to be able to project the values of multiple keys. An extreme version of this would be a summary index that outputs the values for all keys associated with a given query. Need to understand how costly this would be to do.

/cc @svachalek @tregoning @nathfisher

[svachalek]

The UI could potentially offer searching longer time spans in exchange for certain restrictions (prefixed, case-sensitive regexes only) but this forces users to remember better what they are searching for; i.e. it reduces usability to some degree.

Reducing UI "spam" is probably a more feasible direction. Already the UI avoids querying the server for answers that can be determined locally (if the entire list of tags was already downloaded, and the filter text has become more restrictive than the previous search, then the new filter can simply be applied to the previous results). Filter entries are also debounced before querying. Still, forcing an "Enter" press or defaulting to prefixed case-sensitive queries unless a box is checked may reduce it further and don't seem "that" onerous in that the functionality is still there, it just requires a little more deliberation from the user.

I'm actually not a fan of pagination. The current UI doesn't paginate tags and I haven't really had complaints here. With the current 1000 limit, a user would have to manually scroll past 1000 results and keep going. I haven't seen users do this much and can't imagine doing it myself. It's much more practical to add more filters, either another tag or more text.

Potentially more valuable would be to select a smarter set than alphabetical matching -- choose shorter text, perhaps, or matches closer to the beginning of the text, in order to eliminate massive spammy values that fill up the result set.

A count or estimate of the total result size would be useful too, in order to indicate when the current filter text may be wholly inadequate. A list of tags that have massive ordinality could also help; currently the "name" tag is special cased to require at least 3 characters or another tag selection before the UI will query it, as 0-2 characters without another tag is likely to return a list so long that the top of it is effectively noise.