Skip to content

Commit 3f0995a

Browse files
authored
[8.18] [Lens][Table] Fix csv export column sort order (#236673) (#237019)
# Backport This will backport the following commits from `main` to `8.18`: - [[Lens][Table] Fix csv export column sort order (#236673)](#236673) <!--- Backport version: 10.0.2 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Nick Partridge","email":"[email protected]"},"sourceCommit":{"committedDate":"2025-09-30T16:45:47Z","message":"[Lens][Table] Fix csv export column sort order (#236673)\n\n## Summary\n\nWhen a table from a dashboard using the **Download CSV** action, the\ncolumn sort order visible in the table on the dashboard are preserved in\nthe csv output.\n\n\nhttps://github.com/user-attachments/assets/7454a511-150b-45a2-86c5-1cd61bc0191a\n\nFixes #236550\n\n## Details\n\nThe `datatable_fn` is used to set the table of data to the `adapters`,\nwhich is the data used in the csv export action. This data comes in out\nof order with additional `Part of X` columns for formula columns.\nHowever, we do not sort these in before passing to the adapters.\nNormally, with formulas, this is not a problem as the columns are in the\ncorrect order but the\n[`tabify`](https://github.com/elastic/kibana/blob/8d4b0956586284c35db97de2baea49b58476ee2a/src/platform/plugins/shared/data/common/search/tabify/tabify.ts#L24)\nlogic mixes up the column order for formulas.\n\nThe fix is to simply sort the columns in the table before passing it to\nthe `adapters.`\n\n### Checklist\n\n- [x] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios\n- [x] The PR description includes the appropriate Release Notes section,\nand the correct `release_note:*` label is applied per the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n- [x] Review the [backport\nguidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing)\nand apply applicable `backport:*` labels.\n\n## Release Note\n\nFixes a bug in the Lens table in which exporting the table from a\ndashboard, which containg formula columns, can result in a different\ncolumn order than shown on the dashboard.\n\nCo-authored-by: Marco Liberati <[email protected]>","sha":"ef105c5c34b15a30ab5a90c2e1573af62cb692d2","branchLabelMapping":{"^v9.2.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:fix","Team:Visualizations","Feature:Lens","backport:all-open","v9.2.0"],"title":"[Lens][Table] Fix csv export column sort order","number":236673,"url":"https://github.com/elastic/kibana/pull/236673","mergeCommit":{"message":"[Lens][Table] Fix csv export column sort order (#236673)\n\n## Summary\n\nWhen a table from a dashboard using the **Download CSV** action, the\ncolumn sort order visible in the table on the dashboard are preserved in\nthe csv output.\n\n\nhttps://github.com/user-attachments/assets/7454a511-150b-45a2-86c5-1cd61bc0191a\n\nFixes #236550\n\n## Details\n\nThe `datatable_fn` is used to set the table of data to the `adapters`,\nwhich is the data used in the csv export action. This data comes in out\nof order with additional `Part of X` columns for formula columns.\nHowever, we do not sort these in before passing to the adapters.\nNormally, with formulas, this is not a problem as the columns are in the\ncorrect order but the\n[`tabify`](https://github.com/elastic/kibana/blob/8d4b0956586284c35db97de2baea49b58476ee2a/src/platform/plugins/shared/data/common/search/tabify/tabify.ts#L24)\nlogic mixes up the column order for formulas.\n\nThe fix is to simply sort the columns in the table before passing it to\nthe `adapters.`\n\n### Checklist\n\n- [x] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios\n- [x] The PR description includes the appropriate Release Notes section,\nand the correct `release_note:*` label is applied per the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n- [x] Review the [backport\nguidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing)\nand apply applicable `backport:*` labels.\n\n## Release Note\n\nFixes a bug in the Lens table in which exporting the table from a\ndashboard, which containg formula columns, can result in a different\ncolumn order than shown on the dashboard.\n\nCo-authored-by: Marco Liberati <[email protected]>","sha":"ef105c5c34b15a30ab5a90c2e1573af62cb692d2"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.2.0","branchLabelMappingKey":"^v9.2.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/236673","number":236673,"mergeCommit":{"message":"[Lens][Table] Fix csv export column sort order (#236673)\n\n## Summary\n\nWhen a table from a dashboard using the **Download CSV** action, the\ncolumn sort order visible in the table on the dashboard are preserved in\nthe csv output.\n\n\nhttps://github.com/user-attachments/assets/7454a511-150b-45a2-86c5-1cd61bc0191a\n\nFixes #236550\n\n## Details\n\nThe `datatable_fn` is used to set the table of data to the `adapters`,\nwhich is the data used in the csv export action. This data comes in out\nof order with additional `Part of X` columns for formula columns.\nHowever, we do not sort these in before passing to the adapters.\nNormally, with formulas, this is not a problem as the columns are in the\ncorrect order but the\n[`tabify`](https://github.com/elastic/kibana/blob/8d4b0956586284c35db97de2baea49b58476ee2a/src/platform/plugins/shared/data/common/search/tabify/tabify.ts#L24)\nlogic mixes up the column order for formulas.\n\nThe fix is to simply sort the columns in the table before passing it to\nthe `adapters.`\n\n### Checklist\n\n- [x] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios\n- [x] The PR description includes the appropriate Release Notes section,\nand the correct `release_note:*` label is applied per the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n- [x] Review the [backport\nguidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing)\nand apply applicable `backport:*` labels.\n\n## Release Note\n\nFixes a bug in the Lens table in which exporting the table from a\ndashboard, which containg formula columns, can result in a different\ncolumn order than shown on the dashboard.\n\nCo-authored-by: Marco Liberati <[email protected]>","sha":"ef105c5c34b15a30ab5a90c2e1573af62cb692d2"}},{"url":"https://github.com/elastic/kibana/pull/237010","number":237010,"branch":"9.1","state":"OPEN"}]}] BACKPORT-->
1 parent 4866ed4 commit 3f0995a

File tree

2 files changed

+138
-7
lines changed

2 files changed

+138
-7
lines changed
Lines changed: 124 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,124 @@
1+
/*
2+
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
3+
* or more contributor license agreements. Licensed under the Elastic License
4+
* 2.0; you may not use this file except in compliance with the Elastic License
5+
* 2.0.
6+
*/
7+
8+
import { shuffle } from 'lodash';
9+
10+
import { fieldFormatsServiceMock } from '@kbn/field-formats-plugin/public/mocks';
11+
import type {
12+
Datatable,
13+
DefaultInspectorAdapters,
14+
ExecutionContext,
15+
} from '@kbn/expressions-plugin/common';
16+
17+
import { datatableFn } from './datatable_fn';
18+
import type { DatatableArgs } from './datatable';
19+
20+
const context = {
21+
variables: { embeddableTitle: 'title' },
22+
} as unknown as ExecutionContext<DefaultInspectorAdapters>;
23+
24+
const mockFormatFactory = fieldFormatsServiceMock.createStartContract().deserialize;
25+
26+
describe('datatableFn', () => {
27+
function buildTable(): Datatable {
28+
return {
29+
type: 'datatable',
30+
columns: [
31+
{ id: 'bucket1', name: 'bucket1', meta: { type: 'string' } },
32+
{ id: 'bucket2', name: 'bucket2', meta: { type: 'string' } },
33+
{ id: 'bucket3', name: 'bucket3', meta: { type: 'string' } },
34+
{ id: 'metric1', name: 'metric1', meta: { type: 'number' } },
35+
{ id: 'metric2', name: 'metric2', meta: { type: 'number' } },
36+
],
37+
rows: [
38+
{ bucket1: 'A', bucket2: 'D', bucket3: 'X', metric1: 1, metric2: 2 },
39+
{ bucket1: 'A', bucket2: 'D', bucket3: 'Y', metric1: 3, metric2: 4 },
40+
{ bucket1: 'A', bucket2: 'D', bucket3: 'Z', metric1: 5, metric2: 6 },
41+
{ bucket1: 'A', bucket2: 'E', bucket3: 'X', metric1: 7, metric2: 8 },
42+
{ bucket1: 'A', bucket2: 'E', bucket3: 'Y', metric1: 9, metric2: 10 },
43+
{ bucket1: 'A', bucket2: 'E', bucket3: 'Z', metric1: 11, metric2: 12 },
44+
{ bucket1: 'A', bucket2: 'F', bucket3: 'X', metric1: 13, metric2: 14 },
45+
{ bucket1: 'A', bucket2: 'F', bucket3: 'Y', metric1: 15, metric2: 16 },
46+
{ bucket1: 'A', bucket2: 'F', bucket3: 'Z', metric1: 17, metric2: 18 },
47+
{ bucket1: 'B', bucket2: 'D', bucket3: 'X', metric1: 19, metric2: 20 },
48+
{ bucket1: 'B', bucket2: 'D', bucket3: 'Y', metric1: 21, metric2: 22 },
49+
{ bucket1: 'B', bucket2: 'D', bucket3: 'Z', metric1: 23, metric2: 24 },
50+
{ bucket1: 'B', bucket2: 'E', bucket3: 'X', metric1: 25, metric2: 26 },
51+
{ bucket1: 'B', bucket2: 'E', bucket3: 'Y', metric1: 27, metric2: 28 },
52+
{ bucket1: 'B', bucket2: 'E', bucket3: 'Z', metric1: 29, metric2: 30 },
53+
{ bucket1: 'B', bucket2: 'F', bucket3: 'X', metric1: 31, metric2: 32 },
54+
{ bucket1: 'B', bucket2: 'F', bucket3: 'Y', metric1: 33, metric2: 34 },
55+
{ bucket1: 'B', bucket2: 'F', bucket3: 'Z', metric1: 35, metric2: 36 },
56+
{ bucket1: 'C', bucket2: 'D', bucket3: 'X', metric1: 37, metric2: 38 },
57+
{ bucket1: 'C', bucket2: 'D', bucket3: 'Y', metric1: 39, metric2: 40 },
58+
{ bucket1: 'C', bucket2: 'D', bucket3: 'Z', metric1: 41, metric2: 42 },
59+
{ bucket1: 'C', bucket2: 'E', bucket3: 'X', metric1: 43, metric2: 44 },
60+
{ bucket1: 'C', bucket2: 'E', bucket3: 'Y', metric1: 45, metric2: 46 },
61+
{ bucket1: 'C', bucket2: 'E', bucket3: 'Z', metric1: 47, metric2: 48 },
62+
{ bucket1: 'C', bucket2: 'F', bucket3: 'X', metric1: 49, metric2: 50 },
63+
{ bucket1: 'C', bucket2: 'F', bucket3: 'Y', metric1: 51, metric2: 52 },
64+
{ bucket1: 'C', bucket2: 'F', bucket3: 'Z', metric1: 53, metric2: 54 },
65+
],
66+
};
67+
}
68+
69+
function buildArgs(): DatatableArgs {
70+
return {
71+
title: 'Table',
72+
sortingColumnId: undefined,
73+
sortingDirection: 'none',
74+
columns: [
75+
{
76+
type: 'lens_datatable_column',
77+
columnId: 'bucket1',
78+
isTransposed: false,
79+
transposable: false,
80+
},
81+
{
82+
type: 'lens_datatable_column',
83+
columnId: 'bucket2',
84+
isTransposed: false,
85+
transposable: false,
86+
},
87+
{
88+
type: 'lens_datatable_column',
89+
columnId: 'bucket3',
90+
isTransposed: false,
91+
transposable: false,
92+
},
93+
{
94+
type: 'lens_datatable_column',
95+
columnId: 'metric1',
96+
isTransposed: false,
97+
transposable: true,
98+
},
99+
{
100+
type: 'lens_datatable_column',
101+
columnId: 'metric2',
102+
isTransposed: false,
103+
transposable: true,
104+
},
105+
],
106+
};
107+
}
108+
109+
it('should correctly sort columns in table by order of args.columns', async () => {
110+
const table = buildTable();
111+
const shuffledTable: Datatable = {
112+
...table,
113+
columns: shuffle(table.columns),
114+
};
115+
const args = buildArgs();
116+
const result = await datatableFn(() => mockFormatFactory)(shuffledTable, args, context);
117+
118+
const resultColumnIds = result.value.data.columns.map((c) => c.id);
119+
const expectedColumnIds = args.columns.map((c) => c.columnId);
120+
121+
expect(resultColumnIds).toEqual(expectedColumnIds);
122+
expect(resultColumnIds).toEqual(['bucket1', 'bucket2', 'bucket3', 'metric1', 'metric2']);
123+
});
124+
});

x-pack/platform/plugins/shared/lens/common/expressions/datatable/datatable_fn.ts

Lines changed: 14 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -27,12 +27,19 @@ export const datatableFn =
2727
getFormatFactory: (context: ExecutionContext) => FormatFactory | Promise<FormatFactory>
2828
): DatatableExpressionFunction['fn'] =>
2929
async (table, args, context) => {
30+
const columnSortMap = args.columns.reduce((acc, c, i) => acc.set(c.columnId, i), new Map());
31+
const getColumnSort = (id: string) => columnSortMap.get(id) ?? -1;
32+
const sortedTable: Datatable = {
33+
...table,
34+
columns: table.columns.slice().sort((a, b) => getColumnSort(a.id) - getColumnSort(b.id)),
35+
};
36+
3037
if (context?.inspectorAdapters?.tables) {
3138
context.inspectorAdapters.tables.reset();
3239
context.inspectorAdapters.tables.allowCsvExport = true;
3340

3441
const logTable = prepareLogTable(
35-
table,
42+
sortedTable,
3643
[
3744
[
3845
args.columns.map((column) => column.columnId),
@@ -52,20 +59,20 @@ export const datatableFn =
5259
const formatters: Record<string, ReturnType<FormatFactory>> = {};
5360
const formatFactory = await getFormatFactory(context);
5461

55-
table.columns.forEach((column) => {
62+
sortedTable.columns.forEach((column) => {
5663
formatters[column.id] = formatFactory(column.meta?.params);
5764
});
5865

5966
const hasTransposedColumns = args.columns.some((c) => c.isTransposed);
6067
if (hasTransposedColumns) {
6168
// store original shape of data separately
62-
untransposedData = cloneDeep(table);
69+
untransposedData = cloneDeep(sortedTable);
6370
// transposes table and args in-place
64-
transposeTable(args, table, formatters);
71+
transposeTable(args, sortedTable, formatters);
6572

6673
if (context?.inspectorAdapters?.tables) {
6774
const logTransposedTable = prepareLogTable(
68-
table,
75+
sortedTable,
6976
[
7077
[
7178
args.columns.map((column) => column.columnId),
@@ -89,7 +96,7 @@ export const datatableFn =
8996
for (const column of columnsWithSummary) {
9097
column.summaryRowValue = computeSummaryRowForColumn(
9198
column,
92-
table,
99+
sortedTable,
93100
formatters,
94101
formatFactory({ id: 'number' })
95102
);
@@ -99,7 +106,7 @@ export const datatableFn =
99106
type: 'render',
100107
as: 'lens_datatable_renderer',
101108
value: {
102-
data: table,
109+
data: sortedTable,
103110
untransposedData,
104111
syncColors: context.isSyncColorsEnabled?.() ?? false,
105112
args: {

0 commit comments

Comments
 (0)