Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions docs/en/resources/sources/bigquery.md
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,7 @@ sources:
# - "my_dataset_1"
# - "other_project.my_dataset_2"
# impersonateServiceAccount: "[email protected]" # Optional: Service account to impersonate
# maxResultRows: 50 # Optional: Limits the number of rows returned by queries. Defaults to 50.
```

Initialize a BigQuery source that uses the client's access token:
Expand All @@ -140,6 +141,7 @@ sources:
# - "my_dataset_1"
# - "other_project.my_dataset_2"
# impersonateServiceAccount: "[email protected]" # Optional: Service account to impersonate
# maxResultRows: 50 # Optional: Limits the number of rows returned by queries. Defaults to 50.
```

## Reference
Expand All @@ -153,3 +155,4 @@ sources:
| allowedDatasets | []string | false | An optional list of dataset IDs that tools using this source are allowed to access. If provided, any tool operation attempting to access a dataset not in this list will be rejected. To enforce this, two types of operations are also disallowed: 1) Dataset-level operations (e.g., `CREATE SCHEMA`), and 2) operations where table access cannot be statically analyzed (e.g., `EXECUTE IMMEDIATE`, `CREATE PROCEDURE`). If a single dataset is provided, it will be treated as the default for prebuilt tools. |
| useClientOAuth | bool | false | If true, forwards the client's OAuth access token from the "Authorization" header to downstream queries. **Note:** This cannot be used with `writeMode: protected`. |
| impersonateServiceAccount | string | false | Service account email to impersonate when making BigQuery and Dataplex API calls. The authenticated principal must have the `roles/iam.serviceAccountTokenCreator` role on the target service account. [Learn More](https://cloud.google.com/iam/docs/service-account-impersonation) |
| maxResultRows | int | false | The maximum number of rows to return from a query. Defaults to 50. |
1 change: 1 addition & 0 deletions internal/prebuiltconfigs/tools/bigquery.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ sources:
project: ${BIGQUERY_PROJECT}
location: ${BIGQUERY_LOCATION:}
useClientOAuth: ${BIGQUERY_USE_CLIENT_OAUTH:false}
maxResultRows: ${BIGQUERY_MAX_RESULT_ROWS:50}

tools:
analyze_contribution:
Expand Down
6 changes: 5 additions & 1 deletion internal/sources/bigquery/bigquery.go
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,7 @@
AllowedDatasets []string `yaml:"allowedDatasets"`
UseClientOAuth bool `yaml:"useClientOAuth"`
ImpersonateServiceAccount string `yaml:"impersonateServiceAccount"`
MaxResultRows int `yaml:"maxResultRows"`
}

func (r Config) SourceConfigKind() string {
Expand Down Expand Up @@ -119,7 +120,7 @@
Client: client,
RestService: restService,
TokenSource: tokenSource,
MaxQueryResultRows: 50,
MaxQueryResultRows: r.MaxResultRows,
ClientCreator: clientCreator,
}

Expand Down Expand Up @@ -530,6 +531,9 @@

var out []any
for {
if s.MaxQueryResultRows > 0 && len(out) >= s.MaxQueryResultRows {

Check failure on line 534 in internal/sources/bigquery/bigquery.go

View workflow job for this annotation

GitHub Actions / lint

QF1006: could lift into loop condition (staticcheck)
break
}
var val []bigqueryapi.Value
err = it.Next(&val)
if err == iterator.Done {
Expand Down
Loading