Skip to content
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/grafana-sync-check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,13 +18,13 @@ on:
push:
branches: [main]
paths:
- "grafana/**"
- "dashboards/grafana/**"
- "scripts/check_grafana_queries_sync.py"
- ".github/workflows/grafana-sync-check.yml"
pull_request:
branches: [main]
paths:
- "grafana/**"
- "dashboards/grafana/**"
- "scripts/check_grafana_queries_sync.py"
- ".github/workflows/grafana-sync-check.yml"

Expand Down
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,9 @@ deploy/streaming_evaluation/.streaming_evaluation_state.json
._*
grafana/datasource.yaml
grafana/.local/
dashboards/grafana/datasource.yaml
dashboards/grafana/.local/

# Service-account credentials (never commit downloaded JSON keys).
grafana/*.json
dashboards/grafana/*.json
2 changes: 1 addition & 1 deletion dashboard/looker_studio/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -171,7 +171,7 @@ from a signed-in, non-owner, out-of-domain account (anonymous HTTP cannot
observe either). While the template is unavailable, the
`tools/hydrate_dashboard.py` preflight below still validates the table — but
its creation URL copies the same template, so it is not an outage workaround;
the [Grafana dashboard](../../grafana/) and the Looker Agent Analytics block
the [Grafana dashboard](../../dashboards/grafana/) and the Looker Agent Analytics block
(below) remain available.

Looker Studio report parameters are intentionally not used for these values:
Expand Down
2 changes: 1 addition & 1 deletion dashboard/looker_studio/USER_MANUAL.md
Original file line number Diff line number Diff line change
Expand Up @@ -232,7 +232,7 @@ configurator produces.
| What you see | What's happening and what to do |
|---|---|
| The new tab shows loading, not-found, or an error page right after clicking **Create my dashboard** | Google is still provisioning your report copy — this resolves within about 10 seconds. Don't close the tab. If it's still broken after a minute, close it and click the button again. (Exception: the "This report isn't shared with you" dialog below never resolves by waiting or retrying.) |
| The new tab shows **"This report isn't shared with you"** with *Reload* / *Return to report list* / *Go to report template* buttons | The shared dashboard *template* is unavailable to the Google account signed in to that tab — nothing is wrong with your project, dataset, table, access, or the identifiers you entered; they haven't been consulted yet. First check the tab is using the account you intend (Looker Studio uses the browser's default Google account — switch accounts or use a profile signed in to only the right one, then click **Create my dashboard** again). If you're on a work or school account, your own organization's sharing policy can also block receiving Looker Studio assets from outside domains — try a personal account if you can. If the dialog still persists, the template's copy path is blocked: report it on [issue #445](https://github.com/GoogleCloudPlatform/BigQuery-Agent-Analytics-SDK/issues/445) — say only whether your signed-in account is personal or part of an organization; do not post the account's email address. While it's blocked, the [compatibility check](#check-compatibility-before-creating-optional-needs-a-terminal) still validates your table (it is not a workaround — its URL copies the same template), and the repository's [Grafana dashboard](https://github.com/GoogleCloudPlatform/BigQuery-Agent-Analytics-SDK/tree/main/grafana) works independently of Looker Studio. |
| The new tab shows **"This report isn't shared with you"** with *Reload* / *Return to report list* / *Go to report template* buttons | The shared dashboard *template* is unavailable to the Google account signed in to that tab — nothing is wrong with your project, dataset, table, access, or the identifiers you entered; they haven't been consulted yet. First check the tab is using the account you intend (Looker Studio uses the browser's default Google account — switch accounts or use a profile signed in to only the right one, then click **Create my dashboard** again). If you're on a work or school account, your own organization's sharing policy can also block receiving Looker Studio assets from outside domains — try a personal account if you can. If the dialog still persists, the template's copy path is blocked: report it on [issue #445](https://github.com/GoogleCloudPlatform/BigQuery-Agent-Analytics-SDK/issues/445) — say only whether your signed-in account is personal or part of an organization; do not post the account's email address. While it's blocked, the [compatibility check](#check-compatibility-before-creating-optional-needs-a-terminal) still validates your table (it is not a workaround — its URL copies the same template), and the repository's [Grafana dashboard](https://github.com/GoogleCloudPlatform/BigQuery-Agent-Analytics-SDK/tree/main/dashboards/grafana) works independently of Looker Studio. |
| Charts blank or trickling in after opening a page | Normal on a cold load — allow up to 90 seconds. If a chart is still empty after that, widen the date range: your table may have no events in the selected window. |
| Layout looks cut off on the left | Collapse the Looker Studio navigation drawer, and make the window at least 1280 px wide. Phones and narrow tablets aren't supported. |
| Bottom charts clipped on Token Consumption or Latency | You're on a copy created before 2026-07-29, which keeps the old page geometry. Create a fresh copy from the configurator. |
Expand Down
2 changes: 1 addition & 1 deletion dashboard/looker_studio/docs/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -189,7 +189,7 @@ <h3>Save, then secure sharing</h3>
validate your table, but it copies the same template, so it is not
a workaround;
while the template is blocked, use the
<a href="https://github.com/GoogleCloudPlatform/BigQuery-Agent-Analytics-SDK/tree/main/grafana">
<a href="https://github.com/GoogleCloudPlatform/BigQuery-Agent-Analytics-SDK/tree/main/dashboards/grafana">
Grafana dashboard</a>.
</aside>

Expand Down
12 changes: 6 additions & 6 deletions grafana/README.md → dashboards/grafana/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

Visualize BQAA telemetry straight from BigQuery in Grafana.
Works on the free tier of Grafana Cloud. It runs alongside the
[`dashboard_v2/`](../dashboard_v2) React app rather than replacing it — both
[`dashboard_v2/`](../../dashboard_v2) React app rather than replacing it — both
read the same data.

```
Expand All @@ -26,7 +26,7 @@ AI Agent app ──SDK──▶ BigQuery agent_events ──ViewManager──▶
On macOS or Linux with `gcloud auth application-default login` already run:

```bash
python3 grafana/run_local.py --project YOUR_PROJECT --dataset YOUR_DATASET
python3 dashboards/grafana/run_local.py --project YOUR_PROJECT --dataset YOUR_DATASET
```

That downloads a pinned Grafana (cached and checksum-verified after the
Expand All @@ -41,7 +41,7 @@ work by default — the job location is selected automatically; pass
`--processing-location EU` (or a region) to pin it. `--sa-key key.json`
switches to the JWT auth documented below and writes the credential file
with `0600` permissions; `--stop` tears it down; everything generated lives
in the disposable, gitignored `grafana/.local/`. Production setups should
in the disposable, gitignored `dashboards/grafana/.local/`. Production setups should
still follow the full steps below with a scoped service account.

> **Known pitfalls if you run Grafana your own way instead:**
Expand Down Expand Up @@ -77,7 +77,7 @@ still follow the full steps below with a scoped service account.
3. **Keys → Add Key → Create new key → JSON**, and download it.

> **Keep the key out of the repo.** `.gitignore` only covers new `*.json`
> files inside `grafana/`. A key saved anywhere else can be committed by
> files inside `dashboards/grafana/`. A key saved anywhere else can be committed by
> accident.

### 3. Prepare the data
Expand Down Expand Up @@ -138,7 +138,7 @@ form with literal `\n` escapes.

```bash
# Docker
cp grafana/datasource.example.yaml grafana/datasource.yaml
cp dashboards/grafana/datasource.example.yaml dashboards/grafana/datasource.yaml
docker run -d -p 3000:3000 \
-e "GF_INSTALL_PLUGINS=grafana-bigquery-datasource" \
-v /path/to/your/datasource.yaml:/etc/grafana/provisioning/datasources/datasource.yaml \
Expand Down Expand Up @@ -267,7 +267,7 @@ every panel's SQL:
sed -e 's/YOUR_PROJECT_ID/my-gcp-project/g' \
-e 's/YOUR_DATASET_ID/my_demo_dataset/g' \
-e 's/agent_events/my_demo_table/g' \
grafana/bqaa-public-demo.json > grafana/bqaa-public-demo.ready.json
dashboards/grafana/bqaa-public-demo.json > dashboards/grafana/bqaa-public-demo.ready.json
```

The third substitution only matters if your events table is not named
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"title": "BigQuery Agent Analytics",
"uid": "bqaa-dashboard",
"description": "Interactive dashboard for the BigQuery Agent Analytics SDK. Panels query the raw agent_events table and the typed ViewManager views (adk_* by default). Panel SQL is maintained in grafana/queries/*.sql \u2014 treat those files as the source of truth.",
"description": "Interactive dashboard for the BigQuery Agent Analytics SDK. Panels query the raw agent_events table and the typed ViewManager views (adk_* by default). Panel SQL is maintained in dashboards/grafana/queries/*.sql treat those files as the source of truth.",
"tags": [
"bqaa",
"bigquery",
Expand Down Expand Up @@ -444,7 +444,7 @@
"x": 12,
"y": 1
},
"description": "Share of events counted as errors: event_type ending in _ERROR, or a non-NULL error_message, or UPPER(status) = 'ERROR'. See grafana/queries/README.md. Ignores the Event Type filter: the four Overview stats share one query that stays unfiltered so the error rate keeps its global context.",
"description": "Share of events counted as errors: event_type ending in _ERROR, or a non-NULL error_message, or UPPER(status) = 'ERROR'. See dashboards/grafana/queries/README.md. Ignores the Event Type filter: the four Overview stats share one query that stays unfiltered so the error rate keeps its global context.",
"datasource": {
"type": "datasource",
"uid": "-- Dashboard --"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@
],
"title": "BQAA Public Demo",
"uid": "bqaa-public-demo",
"description": "Public-sharing build of bqaa-dashboard.json. Everything a public dashboard cannot evaluate has been removed: templating.list is empty, panel SQL carries no dashboard-variable interpolation, hourly buckets come from BigQuery's own TIMESTAMP_TRUNC instead of Grafana's frontend time-group macros, the cost rates are hardcoded literals, no panel reads another panel's result through the \"-- Dashboard --\" datasource (that resolution happens in the browser and never runs for an anonymous viewer, so such panels render empty), and the per-session Trace detail panel is gone so no raw event timeline is exposed. Every stat therefore issues its own BigQuery query — set a BigQuery Custom Quota on the project before sharing, as grafana/README.md describes. The 72-hour window is enforced in the SQL itself — every panel filters on timestamp >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 72 HOUR) rather than Grafana's time-range macro. A hidden time picker only removes the widget; a from/to pair in the URL still reaches the backend, so a macro-based range would let any viewer widen the BigQuery scan at will. Keep the bound in SQL: it holds no matter how the dashboard is shared or what the URL says. timepicker.refresh_intervals is empty and refresh is off so a viewer cannot pin the dashboard to a fast auto-refresh, editable is false, and timezone is utc to match the UTC buckets TIMESTAMP_TRUNC produces. Before importing, replace the literal YOUR_PROJECT_ID and YOUR_DATASET_ID placeholders in every panel's SQL (see the sed shortcut in grafana/README.md) and confirm adk_ matches your ViewManager view prefix. Do not point this at real production telemetry; seed a dedicated demo dataset instead. Edit bqaa-dashboard.json, not this file, for interactive use.",
"description": "Public-sharing build of bqaa-dashboard.json. Everything a public dashboard cannot evaluate has been removed: templating.list is empty, panel SQL carries no dashboard-variable interpolation, hourly buckets come from BigQuery's own TIMESTAMP_TRUNC instead of Grafana's frontend time-group macros, the cost rates are hardcoded literals, no panel reads another panel's result through the \"-- Dashboard --\" datasource (that resolution happens in the browser and never runs for an anonymous viewer, so such panels render empty), and the per-session Trace detail panel is gone so no raw event timeline is exposed. Every stat therefore issues its own BigQuery query — set a BigQuery Custom Quota on the project before sharing, as dashboards/grafana/README.md describes. The 72-hour window is enforced in the SQL itself — every panel filters on timestamp >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 72 HOUR) rather than Grafana's time-range macro. A hidden time picker only removes the widget; a from/to pair in the URL still reaches the backend, so a macro-based range would let any viewer widen the BigQuery scan at will. Keep the bound in SQL: it holds no matter how the dashboard is shared or what the URL says. timepicker.refresh_intervals is empty and refresh is off so a viewer cannot pin the dashboard to a fast auto-refresh, editable is false, and timezone is utc to match the UTC buckets TIMESTAMP_TRUNC produces. Before importing, replace the literal YOUR_PROJECT_ID and YOUR_DATASET_ID placeholders in every panel's SQL (see the sed shortcut in dashboards/grafana/README.md) and confirm adk_ matches your ViewManager view prefix. Do not point this at real production telemetry; seed a dedicated demo dataset instead. Edit bqaa-dashboard.json, not this file, for interactive use.",
"tags": [
"bqaa",
"bigquery",
Expand Down Expand Up @@ -237,7 +237,7 @@
"x": 12,
"y": 1
},
"description": "Share of events counted as errors: event_type ending in _ERROR, or a non-NULL error_message, or UPPER(status) = 'ERROR'. See grafana/queries/README.md.",
"description": "Share of events counted as errors: event_type ending in _ERROR, or a non-NULL error_message, or UPPER(status) = 'ERROR'. See dashboards/grafana/queries/README.md.",
"datasource": {
"type": "grafana-bigquery-datasource",
"uid": "${DS_BIGQUERY}"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
# OPTIONAL — only for self-managed Grafana (Docker, k8s, bare metal).
# On Grafana Cloud, add the data source through the UI instead
# (Connections -> Data sources -> Add data source -> "Google BigQuery");
# see grafana/README.md.
# see dashboards/grafana/README.md.
#
# Usage (self-managed):
# 1. Install the plugin: grafana-cli plugins install grafana-bigquery-datasource
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Grafana Panel Queries: Source of Truth

The `.sql` files in this directory are the **canonical source of truth** for
every panel and query template variable in `grafana/bqaa-dashboard.json`.
every panel and query template variable in `dashboards/grafana/bqaa-dashboard.json`.
The dashboard JSON embeds a copy of each query (Grafana has no "include SQL
from file" mechanism), so:

Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Public Demo Panel Queries

The `.sql` files here are the **canonical source of truth** for every panel in
`grafana/bqaa-public-demo.json` — one file per panel, embedded as a copy in the
`dashboards/grafana/bqaa-public-demo.json` — one file per panel, embedded as a copy in the
dashboard JSON.

> **Edit the query here first, then paste it into the matching panel in
Expand Down
File renamed without changes.
20 changes: 10 additions & 10 deletions grafana/run_local.py → dashboards/grafana/run_local.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,15 +14,15 @@
# limitations under the License.
"""One-command local Grafana for BigQuery Agent Analytics.

python3 grafana/run_local.py --project MY_PROJECT --dataset MY_DATASET
python3 dashboards/grafana/run_local.py --project MY_PROJECT --dataset MY_DATASET

does everything the manual setup chain in grafana/README.md does: downloads
does everything the manual setup chain in dashboards/grafana/README.md does: downloads
a pinned Grafana, installs the pinned BigQuery datasource plugin, provisions
the datasource (Application Default Credentials by default, --sa-key for the
documented JWT path), writes a copy of bqaa-dashboard.json with the six
constant variables filled in, and launches bound to 127.0.0.1 with the
plugin preinstaller disabled. `--stop` tears it down. Everything generated
lives under grafana/.local/ and is disposable; the committed dashboard and
lives under dashboards/grafana/.local/ and is disposable; the committed dashboard and
example files are never modified.

Requires only the Python standard library. Views (`adk_*`) are created via
Expand Down Expand Up @@ -64,7 +64,7 @@
PLUGIN_VERSION = "3.3.1"
DASHBOARD_UID = "bqaa-dashboard"
DATASOURCE_UID = "bqaa-bigquery"
# Matches grafana/datasource.example.yaml: a per-query BigQuery cost cap of
# Matches dashboards/grafana/datasource.example.yaml: a per-query BigQuery cost cap of
# 100 MB billed. The key spelling is significant (see the example file).
DEFAULT_MAX_BYTES_BILLED = "100000000"

Expand All @@ -78,7 +78,7 @@
# ("us-central1", "asia-northeast1").
LOCATION_RE = re.compile(r"^[A-Za-z][A-Za-z0-9-]{0,31}$")
# The two price constants are interpolated into panel arithmetic
# (grafana/README.md warns a Textbox there is an injection risk), so only a
# (dashboards/grafana/README.md warns a Textbox there is an injection risk), so only a
# strict decimal literal is accepted. Same reasoning for the bytes cap.
PRICE_RE = re.compile(r"^\d{1,9}(\.\d{1,9})?$")
BYTES_RE = re.compile(r"^[1-9]\d{0,17}$")
Expand Down Expand Up @@ -108,7 +108,7 @@ def require_price(label: str, value: str) -> str:
if not PRICE_RE.fullmatch(value):
raise ValueError(
f"{label} {value!r} must be a plain decimal like 1.25 (it is"
" interpolated into panel arithmetic; see grafana/README.md)."
" interpolated into panel arithmetic; see dashboards/grafana/README.md)."
)
return value

Expand Down Expand Up @@ -180,13 +180,13 @@ def render_datasource_yaml(
With no key: `gce` authentication, which the plugin resolves through
Application Default Credentials when Grafana runs off-GCE — no
service-account key needed for local evaluation. With a key: the JWT path
documented in grafana/README.md, with the private key as a real YAML
documented in dashboards/grafana/README.md, with the private key as a real YAML
block scalar (never literal \\n escapes).

`processingLocation` is omitted by default so the plugin selects the job
location automatically; hard-coding a multi-region breaks datasets that
live anywhere else. `MaxBytesBilled` preserves the per-query cost cap
from grafana/datasource.example.yaml.
from dashboards/grafana/datasource.example.yaml.
"""
header = (
"apiVersion: 1\n"
Expand Down Expand Up @@ -324,7 +324,7 @@ def pick_dist(system: str | None = None, machine: str | None = None) -> str:
raise ValueError(
f"unsupported platform {system}/{machine}: this launcher covers"
" macOS and Linux on amd64/arm64; on other platforms follow the"
" manual steps in grafana/README.md."
" manual steps in dashboards/grafana/README.md."
)
return f"{system}-{arch}"

Expand Down Expand Up @@ -657,7 +657,7 @@ def main(argv: list[str] | None = None) -> int:
"--max-bytes-billed",
default=DEFAULT_MAX_BYTES_BILLED,
help="per-query BigQuery cost cap in bytes (default 100000000, matching"
" grafana/datasource.example.yaml)",
" dashboards/grafana/datasource.example.yaml)",
)
parser.add_argument(
"--sa-key",
Expand Down
Loading
Loading