Skip to content

feat(details): Generate Statistics — plugin-native per-column profile - #29

Merged
bcreddy-gcp merged 3 commits into
ui-polishfrom
generate-statistics
Sep 25, 2026
Merged

bcreddy-gcp merged 3 commits into
ui-polishfrom
generate-statistics

Conversation

@bcreddy-gcp

Copy link
Copy Markdown
Collaborator

Summary

Adds a Generate Statistics action to Table Details that computes a per-column profile server-side (one aggregation query) and shows it in an on-demand Statistics tab — a plugin-native replacement for beatrix's notebook %bigquery_stats magic, which was never supported in Workbench.

What it captures (per top-level scalar column)

  • Null count + %
  • Exact distinct count + % (COUNT(DISTINCT))
  • Numeric columns: min / max / avg / stddev, plus zero and negative counts (+%); float columns add an infinite count
  • Nested (RECORD/STRUCT) and REPEATED columns are skipped and listed
  • Summary line shows rows + bytes processed

Top values (opt-in toggle)

"Show top values (approximate)" adds a per-column value distribution via APPROX_TOP_COUNT, rendered as mini bar charts. It stays within the same single query (no per-column fan-out), so the default profile remains cheap.

Cost

The profile scans the selected columns, so it is billable — it runs only on the explicit action, and the UI surfaces bytes processed.

Implementation

  • Backend services/stats.py (table_stats) + TableStatsHandler route tableStats (with topValues arg).
  • Frontend api.tableStats / IColumnStats + StatisticsTab in TableDetails.tsx; CSS for the table, toggle and mini charts.

Tests

  • pytest 42/42 (new test_stats.py: exact distinct, numeric counts, float-only infinite, top-values toggle, skip nested)
  • jest 67/67 (new: on-demand profile renders; top-values toggle re-requests with top-N)
  • tsc + prettier clean; labextension builds

Notes

BUG=557344886

Add a "Generate Statistics" action to Table Details that computes a
per-column profile server-side (one aggregation query) and shows it in an
on-demand Statistics tab — replacing beatrix's notebook `%bigquery_stats`
magic, which was never supported in Workbench.

Per scalar column: null count/%, exact distinct count/% (COUNT(DISTINCT)).
Numeric columns add min/max/avg/stddev and zero/negative counts; float
columns add an infinite count. Nested/repeated columns are skipped and
listed. A "Show top values" toggle adds an approximate per-column value
distribution (APPROX_TOP_COUNT) rendered as mini bar charts — still one
query, opt-in.

Backend services/stats.py + tableStats route; frontend api.tableStats +
StatisticsTab. Tests: pytest 42/42 (test_stats.py), jest 67/67.

BUG=557344886
Add an "ⓘ" affordance and a title tooltip on the "Show top values"
toggle clarifying that it estimates each column's most frequent values
via APPROX_TOP_COUNT (approximate, not exact) within the same single
query — the behavior wasn't self-evident from the label alone.
…ole-style Details

- Generate Statistics ran the profiling query in the TABLE's project, so
  profiling bigquery-public-data failed with 'User does not have
  bigquery.jobs.create'. Run it in the caller's billing project instead
  (plumbed from the app default project; ADC default as fallback). Table
  metadata is still read cross-project.
- Explorer 'Query history' footer is now position:sticky so it stays visible
  while a long project tree scrolls (was unreachable under bigquery-public-data).
- Table Details styled like the BigQuery console: Description + Labels shown
  side by side, zebra-striped Table info, shaded Schema header. Backend
  /table now returns labels.

Tests: pytest 43/43 (billing-project + labels), jest 67/67.
@bcreddy-gcp
bcreddy-gcp merged commit 8dc4ba5 into ui-polish Sep 25, 2026
4 checks passed
bcreddy-gcp added a commit that referenced this pull request Sep 25, 2026
feat(details): re-land Generate Statistics (stranded from #29)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants