Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/grafana-sync-check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,13 +18,13 @@ on:
push:
branches: [main]
paths:
- "grafana/**"
- "dashboards/grafana/**"
- "scripts/check_grafana_queries_sync.py"
- ".github/workflows/grafana-sync-check.yml"
pull_request:
branches: [main]
paths:
- "grafana/**"
- "dashboards/grafana/**"
- "scripts/check_grafana_queries_sync.py"
- ".github/workflows/grafana-sync-check.yml"

Expand Down
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,9 @@ deploy/streaming_evaluation/.streaming_evaluation_state.json
._*
grafana/datasource.yaml
grafana/.local/
dashboards/grafana/datasource.yaml
dashboards/grafana/.local/

# Service-account credentials (never commit downloaded JSON keys).
grafana/*.json
dashboards/grafana/*.json
2 changes: 1 addition & 1 deletion dashboard/looker_studio/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -171,7 +171,7 @@ from a signed-in, non-owner, out-of-domain account (anonymous HTTP cannot
observe either). While the template is unavailable, the
`tools/hydrate_dashboard.py` preflight below still validates the table — but
its creation URL copies the same template, so it is not an outage workaround;
the [Grafana dashboard](../../grafana/) and the Looker Agent Analytics block
the [Grafana dashboard](../../dashboards/grafana/) and the Looker Agent Analytics block
(below) remain available.

Looker Studio report parameters are intentionally not used for these values:
Expand Down
2 changes: 1 addition & 1 deletion dashboard/looker_studio/USER_MANUAL.md
Original file line number Diff line number Diff line change
Expand Up @@ -232,7 +232,7 @@ configurator produces.
| What you see | What's happening and what to do |
|---|---|
| The new tab shows loading, not-found, or an error page right after clicking **Create my dashboard** | Google is still provisioning your report copy — this resolves within about 10 seconds. Don't close the tab. If it's still broken after a minute, close it and click the button again. (Exception: the "This report isn't shared with you" dialog below never resolves by waiting or retrying.) |
| The new tab shows **"This report isn't shared with you"** with *Reload* / *Return to report list* / *Go to report template* buttons | The shared dashboard *template* is unavailable to the Google account signed in to that tab — nothing is wrong with your project, dataset, table, access, or the identifiers you entered; they haven't been consulted yet. First check the tab is using the account you intend (Looker Studio uses the browser's default Google account — switch accounts or use a profile signed in to only the right one, then click **Create my dashboard** again). If you're on a work or school account, your own organization's sharing policy can also block receiving Looker Studio assets from outside domains — try a personal account if you can. If the dialog still persists, the template's copy path is blocked: report it on [issue #445](https://github.com/GoogleCloudPlatform/BigQuery-Agent-Analytics-SDK/issues/445) — say only whether your signed-in account is personal or part of an organization; do not post the account's email address. While it's blocked, the [compatibility check](#check-compatibility-before-creating-optional-needs-a-terminal) still validates your table (it is not a workaround — its URL copies the same template), and the repository's [Grafana dashboard](https://github.com/GoogleCloudPlatform/BigQuery-Agent-Analytics-SDK/tree/main/grafana) works independently of Looker Studio. |
| The new tab shows **"This report isn't shared with you"** with *Reload* / *Return to report list* / *Go to report template* buttons | The shared dashboard *template* is unavailable to the Google account signed in to that tab — nothing is wrong with your project, dataset, table, access, or the identifiers you entered; they haven't been consulted yet. First check the tab is using the account you intend (Looker Studio uses the browser's default Google account — switch accounts or use a profile signed in to only the right one, then click **Create my dashboard** again). If you're on a work or school account, your own organization's sharing policy can also block receiving Looker Studio assets from outside domains — try a personal account if you can. If the dialog still persists, the template's copy path is blocked: report it on [issue #445](https://github.com/GoogleCloudPlatform/BigQuery-Agent-Analytics-SDK/issues/445) — say only whether your signed-in account is personal or part of an organization; do not post the account's email address. While it's blocked, the [compatibility check](#check-compatibility-before-creating-optional-needs-a-terminal) still validates your table (it is not a workaround — its URL copies the same template), and the repository's [Grafana dashboard](https://github.com/GoogleCloudPlatform/BigQuery-Agent-Analytics-SDK/tree/main/dashboards/grafana) works independently of Looker Studio. |
| Charts blank or trickling in after opening a page | Normal on a cold load — allow up to 90 seconds. If a chart is still empty after that, widen the date range: your table may have no events in the selected window. |
| Layout looks cut off on the left | Collapse the Looker Studio navigation drawer, and make the window at least 1280 px wide. Phones and narrow tablets aren't supported. |
| Bottom charts clipped on Token Consumption or Latency | You're on a copy created before 2026-07-29, which keeps the old page geometry. Create a fresh copy from the configurator. |
Expand Down
2 changes: 1 addition & 1 deletion dashboard/looker_studio/docs/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -189,7 +189,7 @@ <h3>Save, then secure sharing</h3>
validate your table, but it copies the same template, so it is not
a workaround;
while the template is blocked, use the
<a href="https://github.com/GoogleCloudPlatform/BigQuery-Agent-Analytics-SDK/tree/main/grafana">
<a href="https://github.com/GoogleCloudPlatform/BigQuery-Agent-Analytics-SDK/tree/main/dashboards/grafana">
Grafana dashboard</a>.
</aside>

Expand Down
12 changes: 6 additions & 6 deletions grafana/README.md → dashboards/grafana/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

Visualize BQAA telemetry straight from BigQuery in Grafana.
Works on the free tier of Grafana Cloud. It runs alongside the
[`dashboard_v2/`](../dashboard_v2) React app rather than replacing it — both
[`dashboard_v2/`](../../dashboard_v2) React app rather than replacing it — both
read the same data.

```
Expand All @@ -26,7 +26,7 @@ AI Agent app ──SDK──▶ BigQuery agent_events ──ViewManager──▶
On macOS or Linux with `gcloud auth application-default login` already run:

```bash
python3 grafana/run_local.py --project YOUR_PROJECT --dataset YOUR_DATASET
python3 dashboards/grafana/run_local.py --project YOUR_PROJECT --dataset YOUR_DATASET
```

That downloads a pinned Grafana (cached and checksum-verified after the
Expand All @@ -41,7 +41,7 @@ work by default — the job location is selected automatically; pass
`--processing-location EU` (or a region) to pin it. `--sa-key key.json`
switches to the JWT auth documented below and writes the credential file
with `0600` permissions; `--stop` tears it down; everything generated lives
in the disposable, gitignored `grafana/.local/`. Production setups should
in the disposable, gitignored `dashboards/grafana/.local/`. Production setups should
still follow the full steps below with a scoped service account.

> **Known pitfalls if you run Grafana your own way instead:**
Expand Down Expand Up @@ -77,7 +77,7 @@ still follow the full steps below with a scoped service account.
3. **Keys → Add Key → Create new key → JSON**, and download it.

> **Keep the key out of the repo.** `.gitignore` only covers new `*.json`
> files inside `grafana/`. A key saved anywhere else can be committed by
> files inside `dashboards/grafana/`. A key saved anywhere else can be committed by
> accident.

### 3. Prepare the data
Expand Down Expand Up @@ -138,7 +138,7 @@ form with literal `\n` escapes.

```bash
# Docker
cp grafana/datasource.example.yaml grafana/datasource.yaml
cp dashboards/grafana/datasource.example.yaml dashboards/grafana/datasource.yaml
docker run -d -p 3000:3000 \
-e "GF_INSTALL_PLUGINS=grafana-bigquery-datasource" \
-v /path/to/your/datasource.yaml:/etc/grafana/provisioning/datasources/datasource.yaml \
Expand Down Expand Up @@ -267,7 +267,7 @@ every panel's SQL:
sed -e 's/YOUR_PROJECT_ID/my-gcp-project/g' \
-e 's/YOUR_DATASET_ID/my_demo_dataset/g' \
-e 's/agent_events/my_demo_table/g' \
grafana/bqaa-public-demo.json > grafana/bqaa-public-demo.ready.json
dashboards/grafana/bqaa-public-demo.json > dashboards/grafana/bqaa-public-demo.ready.json
```

The third substitution only matters if your events table is not named
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"title": "BigQuery Agent Analytics",
"uid": "bqaa-dashboard",
"description": "Interactive dashboard for the BigQuery Agent Analytics SDK. Panels query the raw agent_events table and the typed ViewManager views (adk_* by default). Panel SQL is maintained in grafana/queries/*.sql \u2014 treat those files as the source of truth.",
"description": "Interactive dashboard for the BigQuery Agent Analytics SDK. Panels query the raw agent_events table and the typed ViewManager views (adk_* by default). Panel SQL is maintained in dashboards/grafana/queries/*.sql treat those files as the source of truth.",
"tags": [
"bqaa",
"bigquery",
Expand Down Expand Up @@ -444,7 +444,7 @@
"x": 12,
"y": 1
},
"description": "Share of events counted as errors: event_type ending in _ERROR, or a non-NULL error_message, or UPPER(status) = 'ERROR'. See grafana/queries/README.md. Ignores the Event Type filter: the four Overview stats share one query that stays unfiltered so the error rate keeps its global context.",
"description": "Share of events counted as errors: event_type ending in _ERROR, or a non-NULL error_message, or UPPER(status) = 'ERROR'. See dashboards/grafana/queries/README.md. Ignores the Event Type filter: the four Overview stats share one query that stays unfiltered so the error rate keeps its global context.",
"datasource": {
"type": "datasource",
"uid": "-- Dashboard --"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@
],
"title": "BQAA Public Demo",
"uid": "bqaa-public-demo",
"description": "Public-sharing build of bqaa-dashboard.json. Everything a public dashboard cannot evaluate has been removed: templating.list is empty, panel SQL carries no dashboard-variable interpolation, hourly buckets come from BigQuery's own TIMESTAMP_TRUNC instead of Grafana's frontend time-group macros, the cost rates are hardcoded literals, no panel reads another panel's result through the \"-- Dashboard --\" datasource (that resolution happens in the browser and never runs for an anonymous viewer, so such panels render empty), and the per-session Trace detail panel is gone so no raw event timeline is exposed. Every stat therefore issues its own BigQuery query — set a BigQuery Custom Quota on the project before sharing, as grafana/README.md describes. The 72-hour window is enforced in the SQL itself — every panel filters on timestamp >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 72 HOUR) rather than Grafana's time-range macro. A hidden time picker only removes the widget; a from/to pair in the URL still reaches the backend, so a macro-based range would let any viewer widen the BigQuery scan at will. Keep the bound in SQL: it holds no matter how the dashboard is shared or what the URL says. timepicker.refresh_intervals is empty and refresh is off so a viewer cannot pin the dashboard to a fast auto-refresh, editable is false, and timezone is utc to match the UTC buckets TIMESTAMP_TRUNC produces. Before importing, replace the literal YOUR_PROJECT_ID and YOUR_DATASET_ID placeholders in every panel's SQL (see the sed shortcut in grafana/README.md) and confirm adk_ matches your ViewManager view prefix. Do not point this at real production telemetry; seed a dedicated demo dataset instead. Edit bqaa-dashboard.json, not this file, for interactive use.",
"description": "Public-sharing build of bqaa-dashboard.json. Everything a public dashboard cannot evaluate has been removed: templating.list is empty, panel SQL carries no dashboard-variable interpolation, hourly buckets come from BigQuery's own TIMESTAMP_TRUNC instead of Grafana's frontend time-group macros, the cost rates are hardcoded literals, no panel reads another panel's result through the \"-- Dashboard --\" datasource (that resolution happens in the browser and never runs for an anonymous viewer, so such panels render empty), and the per-session Trace detail panel is gone so no raw event timeline is exposed. Every stat therefore issues its own BigQuery query — set a BigQuery Custom Quota on the project before sharing, as dashboards/grafana/README.md describes. The 72-hour window is enforced in the SQL itself — every panel filters on timestamp >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 72 HOUR) rather than Grafana's time-range macro. A hidden time picker only removes the widget; a from/to pair in the URL still reaches the backend, so a macro-based range would let any viewer widen the BigQuery scan at will. Keep the bound in SQL: it holds no matter how the dashboard is shared or what the URL says. timepicker.refresh_intervals is empty and refresh is off so a viewer cannot pin the dashboard to a fast auto-refresh, editable is false, and timezone is utc to match the UTC buckets TIMESTAMP_TRUNC produces. Before importing, replace the literal YOUR_PROJECT_ID and YOUR_DATASET_ID placeholders in every panel's SQL (see the sed shortcut in dashboards/grafana/README.md) and confirm adk_ matches your ViewManager view prefix. Do not point this at real production telemetry; seed a dedicated demo dataset instead. Edit bqaa-dashboard.json, not this file, for interactive use.",
"tags": [
"bqaa",
"bigquery",
Expand Down Expand Up @@ -237,7 +237,7 @@
"x": 12,
"y": 1
},
"description": "Share of events counted as errors: event_type ending in _ERROR, or a non-NULL error_message, or UPPER(status) = 'ERROR'. See grafana/queries/README.md.",
"description": "Share of events counted as errors: event_type ending in _ERROR, or a non-NULL error_message, or UPPER(status) = 'ERROR'. See dashboards/grafana/queries/README.md.",
"datasource": {
"type": "grafana-bigquery-datasource",
"uid": "${DS_BIGQUERY}"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
# OPTIONAL — only for self-managed Grafana (Docker, k8s, bare metal).
# On Grafana Cloud, add the data source through the UI instead
# (Connections -> Data sources -> Add data source -> "Google BigQuery");
# see grafana/README.md.
# see dashboards/grafana/README.md.
#
# Usage (self-managed):
# 1. Install the plugin: grafana-cli plugins install grafana-bigquery-datasource
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Grafana Panel Queries: Source of Truth

The `.sql` files in this directory are the **canonical source of truth** for
every panel and query template variable in `grafana/bqaa-dashboard.json`.
every panel and query template variable in `dashboards/grafana/bqaa-dashboard.json`.
The dashboard JSON embeds a copy of each query (Grafana has no "include SQL
from file" mechanism), so:

Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Public Demo Panel Queries

The `.sql` files here are the **canonical source of truth** for every panel in
`grafana/bqaa-public-demo.json` — one file per panel, embedded as a copy in the
`dashboards/grafana/bqaa-public-demo.json` — one file per panel, embedded as a copy in the
dashboard JSON.

> **Edit the query here first, then paste it into the matching panel in
Expand Down
File renamed without changes.
20 changes: 10 additions & 10 deletions grafana/run_local.py → dashboards/grafana/run_local.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,15 +14,15 @@
# limitations under the License.
"""One-command local Grafana for BigQuery Agent Analytics.

python3 grafana/run_local.py --project MY_PROJECT --dataset MY_DATASET
python3 dashboards/grafana/run_local.py --project MY_PROJECT --dataset MY_DATASET

does everything the manual setup chain in grafana/README.md does: downloads
does everything the manual setup chain in dashboards/grafana/README.md does: downloads
a pinned Grafana, installs the pinned BigQuery datasource plugin, provisions
the datasource (Application Default Credentials by default, --sa-key for the
documented JWT path), writes a copy of bqaa-dashboard.json with the six
constant variables filled in, and launches bound to 127.0.0.1 with the
plugin preinstaller disabled. `--stop` tears it down. Everything generated
lives under grafana/.local/ and is disposable; the committed dashboard and
lives under dashboards/grafana/.local/ and is disposable; the committed dashboard and
example files are never modified.

Requires only the Python standard library. Views (`adk_*`) are created via
Expand Down Expand Up @@ -64,7 +64,7 @@
PLUGIN_VERSION = "3.3.1"
DASHBOARD_UID = "bqaa-dashboard"
DATASOURCE_UID = "bqaa-bigquery"
# Matches grafana/datasource.example.yaml: a per-query BigQuery cost cap of
# Matches dashboards/grafana/datasource.example.yaml: a per-query BigQuery cost cap of
# 100 MB billed. The key spelling is significant (see the example file).
DEFAULT_MAX_BYTES_BILLED = "100000000"

Expand All @@ -78,7 +78,7 @@
# ("us-central1", "asia-northeast1").
LOCATION_RE = re.compile(r"^[A-Za-z][A-Za-z0-9-]{0,31}$")
# The two price constants are interpolated into panel arithmetic
# (grafana/README.md warns a Textbox there is an injection risk), so only a
# (dashboards/grafana/README.md warns a Textbox there is an injection risk), so only a
# strict decimal literal is accepted. Same reasoning for the bytes cap.
PRICE_RE = re.compile(r"^\d{1,9}(\.\d{1,9})?$")
BYTES_RE = re.compile(r"^[1-9]\d{0,17}$")
Expand Down Expand Up @@ -108,7 +108,7 @@ def require_price(label: str, value: str) -> str:
if not PRICE_RE.fullmatch(value):
raise ValueError(
f"{label} {value!r} must be a plain decimal like 1.25 (it is"
" interpolated into panel arithmetic; see grafana/README.md)."
" interpolated into panel arithmetic; see dashboards/grafana/README.md)."
)
return value

Expand Down Expand Up @@ -180,13 +180,13 @@ def render_datasource_yaml(
With no key: `gce` authentication, which the plugin resolves through
Application Default Credentials when Grafana runs off-GCE — no
service-account key needed for local evaluation. With a key: the JWT path
documented in grafana/README.md, with the private key as a real YAML
documented in dashboards/grafana/README.md, with the private key as a real YAML
block scalar (never literal \\n escapes).

`processingLocation` is omitted by default so the plugin selects the job
location automatically; hard-coding a multi-region breaks datasets that
live anywhere else. `MaxBytesBilled` preserves the per-query cost cap
from grafana/datasource.example.yaml.
from dashboards/grafana/datasource.example.yaml.
"""
header = (
"apiVersion: 1\n"
Expand Down Expand Up @@ -324,7 +324,7 @@ def pick_dist(system: str | None = None, machine: str | None = None) -> str:
raise ValueError(
f"unsupported platform {system}/{machine}: this launcher covers"
" macOS and Linux on amd64/arm64; on other platforms follow the"
" manual steps in grafana/README.md."
" manual steps in dashboards/grafana/README.md."
)
return f"{system}-{arch}"

Expand Down Expand Up @@ -657,7 +657,7 @@ def main(argv: list[str] | None = None) -> int:
"--max-bytes-billed",
default=DEFAULT_MAX_BYTES_BILLED,
help="per-query BigQuery cost cap in bytes (default 100000000, matching"
" grafana/datasource.example.yaml)",
" dashboards/grafana/datasource.example.yaml)",
)
parser.add_argument(
"--sa-key",
Expand Down
Loading
Loading